[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:02:46 dist-test-slave-nfb5 krb5kdc[25235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:02:46 dist-test-slave-nfb5 krb5kdc[25235](info): set up 2 sockets
Aug 08 10:02:46 dist-test-slave-nfb5 krb5kdc[25235](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 10:02:48.122107 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.380s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:02:48.249734 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38777
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:48.336694 25256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:48.336901 25256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:48.336925 25256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:48.338075 25256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:02:48.338120 25256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:48.338138 25256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:48.338192 25256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:02:48.338212 25256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:02:48.339978 25256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:38777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:48.340359 25256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:48.340662 25256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:48.348313 25263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.348382 25262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.348677 25266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:48.349280 25256 server_base.cc:1034] running on GCE node
I20260808 10:02:48.349582 25256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:48.349983 25256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:48.351166 25256 hybrid_clock.cc:648] HybridClock initialized: now 1786183368351143 us; error 78 us; skew 500 ppm
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:48.353799 25256 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:02:48.354463 25256 webserver.cc:460] Webserver started at http://127.24.162.190:46425/ using document root <none> and password file <none>
I20260808 10:02:48.354756 25256 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:48.354804 25256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:48.355002 25256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:48.356364 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "fc827d5104fd4a9ab697b1cc84c0992c"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "8517c8a8e7fbe26d6f2094328cd62f05"
server_key_iv: "f8b0aa8ee2910a0535708bb287d63861"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.356997 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "fc827d5104fd4a9ab697b1cc84c0992c"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "8517c8a8e7fbe26d6f2094328cd62f05"
server_key_iv: "f8b0aa8ee2910a0535708bb287d63861"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.358834 25256 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 10:02:48.359932 25275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.360229 25256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:48.360332 25256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "fc827d5104fd4a9ab697b1cc84c0992c"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "8517c8a8e7fbe26d6f2094328cd62f05"
server_key_iv: "f8b0aa8ee2910a0535708bb287d63861"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.360404 25256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:48.371948 25256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:48.374595 25256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:48.374787 25256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:48.381603 25256 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:38777
I20260808 10:02:48.381657 25355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:38777 every 8 connection(s)
I20260808 10:02:48.382208 25256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:02:48.383612 25356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:48.384826 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25256
I20260808 10:02:48.384910 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:02:48.385551 25226 external_mini_cluster.cc:1428] Setting key af3de282cdd1c847450abe18a6fc052f
I20260808 10:02:48.387470 25356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c: Bootstrap starting.
I20260808 10:02:48.388994 25356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:48.389987 25356 log.cc:824] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:48.390905 25356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c: No bootstrap required, opened a new log
I20260808 10:02:48.393450 25356 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } }
I20260808 10:02:48.393671 25356 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:48.393801 25356 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc827d5104fd4a9ab697b1cc84c0992c, State: Initialized, Role: FOLLOWER
I20260808 10:02:48.394019 25356 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } }
I20260808 10:02:48.394181 25356 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:02:48.394229 25356 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:02:48.394280 25356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:48.395596 25356 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } }
I20260808 10:02:48.395803 25356 leader_election.cc:302] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc827d5104fd4a9ab697b1cc84c0992c; no voters:
I20260808 10:02:48.396015 25356 leader_election.cc:288] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:02:48.396085 25365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:48.396271 25365 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [term 1 LEADER]: Becoming Leader. State: Replica: fc827d5104fd4a9ab697b1cc84c0992c, State: Running, Role: LEADER
I20260808 10:02:48.396425 25365 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } }
I20260808 10:02:48.396543 25356 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:02:48.397015 25365 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc827d5104fd4a9ab697b1cc84c0992c. Latest consensus state: current_term: 1 leader_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } } }
I20260808 10:02:48.397099 25365 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:48.397094 25370 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc827d5104fd4a9ab697b1cc84c0992c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38777 } } }
I20260808 10:02:48.397183 25370 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:48.398089 25377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:02:48.399438 25377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:02:48.401616 25377 catalog_manager.cc:1347] Generated new cluster ID: 980cdd6755514cdf9f9431171a96b153
I20260808 10:02:48.401669 25377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:48.409852 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.386828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44440 (local address 127.24.162.190:38777)
0808 10:02:48.387152 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:48.387163 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:48.389472 (+ 2309us) server_negotiation.cc:407] Connection header received
0808 10:02:48.392628 (+ 3156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:48.392637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:48.392780 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:48.392945 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:48.394887 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.396589 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.398287 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.399015 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.399177 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.404788 (+ 5611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:48.404793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:48.404793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:48.404806 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:48.407170 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.407688 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.407693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.407693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.407822 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.408120 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.408121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.408121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.408538 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:48.408708 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:48.409029 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 10:02:48.409235 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":102,"threads_started":1}
I20260808 10:02:48.412374 25377 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:02:48.413136 25377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:02:48.420569 25377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc827d5104fd4a9ab697b1cc84c0992c: Generated new TSK 0
I20260808 10:02:48.420769 25377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:02:48.458356 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:48.582589 25396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:48.582844 25396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:48.582885 25396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:48.584110 25396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:48.584183 25396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:48.584231 25396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:02:48.585850 25396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:48.586189 25396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:48.586427 25396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:48.592723 25405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.592783 25408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.592878 25404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:48.593374 25396 server_base.cc:1034] running on GCE node
I20260808 10:02:48.593601 25396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:48.593989 25396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:48.595167 25396 hybrid_clock.cc:648] HybridClock initialized: now 1786183368595143 us; error 42 us; skew 500 ppm
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:48.598155 25396 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:02:48.598608 25396 webserver.cc:460] Webserver started at http://127.24.162.129:37243/ using document root <none> and password file <none>
I20260808 10:02:48.598868 25396 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:48.598964 25396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:48.599128 25396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:48.600178 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "1087ae0e80b443ca916d28499fb5275a"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "795f8c0832617075e92b51eba926f623"
server_key_iv: "77486b1efc43a630890d25ca2341c792"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.600591 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "1087ae0e80b443ca916d28499fb5275a"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "795f8c0832617075e92b51eba926f623"
server_key_iv: "77486b1efc43a630890d25ca2341c792"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.602164 25396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:02:48.603127 25417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.603344 25396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:48.603456 25396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "1087ae0e80b443ca916d28499fb5275a"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "795f8c0832617075e92b51eba926f623"
server_key_iv: "77486b1efc43a630890d25ca2341c792"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.603539 25396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:48.632239 25396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:48.634245 25396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:48.634426 25396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:48.634727 25396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:48.636742 25396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:48.636857 25396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.636898 25396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:48.636947 25396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.649205 25396 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:38549
I20260808 10:02:48.649286 25572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:38549 every 8 connection(s)
I20260808 10:02:48.649626 25396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:02:48.655408 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25396
I20260808 10:02:48.655517 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:02:48.655685 25226 external_mini_cluster.cc:1428] Setting key 5375a622184b5a5fc3017bc1830cdc09
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:48.661232 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.650597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:56101 (local address 127.24.162.190:38777)
0808 10:02:48.650649 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:48.650651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:48.651032 (+ 381us) server_negotiation.cc:407] Connection header received
0808 10:02:48.651770 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:48.651772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:48.651784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:48.651842 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:48.652919 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.653439 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.654434 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.654639 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.654729 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.657827 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:48.657829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:48.657829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:48.657838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:48.659588 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.660226 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.660227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.660228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.660284 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.660574 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.660574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.660574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.660706 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:48.660761 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:48.661086 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 10:02:48.661152 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:02:48.661628 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.650542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38777 (local address 127.24.162.129:56101)
0808 10:02:48.650859 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:48.650897 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:48.651649 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:48.651921 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:48.651927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:48.652608 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:48.652800 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.652805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.653542 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.653543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.654339 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.654341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.654743 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.654744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.654874 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.655596 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:48.655606 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:48.657726 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:48.659675 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.659729 (+ 54us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.659731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.660107 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.660356 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.660356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.660357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.660505 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.660775 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:48.660781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:48.660949 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 10:02:48.661067 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 10:02:48.661261 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":172,"threads_started":1}
I20260808 10:02:48.662341 25573 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38777
I20260808 10:02:48.662452 25573 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:48.662662 25573 heartbeater.cc:511] Master 127.24.162.190:38777 requested a full tablet report, sending...
I20260808 10:02:48.663636 25302 ts_manager.cc:194] Registered new tserver with Master: 1087ae0e80b443ca916d28499fb5275a (127.24.162.129:38549)
I20260808 10:02:48.664412 25302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:56101
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:02:48.687891 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:48.783757 25583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:48.783953 25583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:48.783977 25583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:48.785036 25583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:48.785072 25583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:48.785099 25583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:02:48.786484 25583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:48.786672 25583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:48.786847 25583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:48.792176 25590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.792055 25591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.792065 25596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:48.792963 25583 server_base.cc:1034] running on GCE node
I20260808 10:02:48.793190 25583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:48.793504 25583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:48.794729 25583 hybrid_clock.cc:648] HybridClock initialized: now 1786183368794713 us; error 41 us; skew 500 ppm
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:48.799000 25583 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:02:48.799548 25583 webserver.cc:460] Webserver started at http://127.24.162.130:41381/ using document root <none> and password file <none>
I20260808 10:02:48.799870 25583 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:48.799942 25583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:48.800082 25583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:48.801615 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "35a7d4add4414c549fef659f6884b2b5"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "1991f68f6e6549db83bc761345d910bb"
server_key_iv: "77c2f4fc6a71905a05a129dc71422f63"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.802028 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "35a7d4add4414c549fef659f6884b2b5"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "1991f68f6e6549db83bc761345d910bb"
server_key_iv: "77c2f4fc6a71905a05a129dc71422f63"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.803498 25583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 10:02:48.804302 25606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.804509 25583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:48.804584 25583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "35a7d4add4414c549fef659f6884b2b5"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "1991f68f6e6549db83bc761345d910bb"
server_key_iv: "77c2f4fc6a71905a05a129dc71422f63"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.804636 25583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:48.813915 25583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:48.815727 25583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:48.815843 25583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:48.816016 25583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:48.816318 25583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:48.816350 25583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.816371 25583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:48.816421 25583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.823506 25583 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:45753
I20260808 10:02:48.823583 25762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:45753 every 8 connection(s)
I20260808 10:02:48.823890 25583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:48.832465 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.824703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:39993 (local address 127.24.162.190:38777)
0808 10:02:48.824771 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:48.824773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:48.824994 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:02:48.825445 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:48.825446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:48.825456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:48.825510 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:48.826381 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.826869 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.827848 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.828031 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.828082 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.830078 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:48.830079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:48.830080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:48.830087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:48.831351 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.831703 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.831704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.831704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.831754 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.831966 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.831967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.831967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.832065 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:48.832116 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:48.832337 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:02:48.832399 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:02:48.832634 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.824680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38777 (local address 127.24.162.130:39993)
0808 10:02:48.824877 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:48.824912 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:48.825367 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:48.825559 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:48.825561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:48.826123 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:48.826309 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.826313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.826988 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.826989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.827754 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.827756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.828105 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.828105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.828200 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.828619 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:48.828627 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:48.829990 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:48.831421 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.831427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.831428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.831637 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.831817 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.831817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.831818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.831912 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.832120 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:48.832124 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:48.832226 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 10:02:48.832305 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:02:48.832416 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260808 10:02:48.833212 25763 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38777
I20260808 10:02:48.833323 25763 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:48.833500 25763 heartbeater.cc:511] Master 127.24.162.190:38777 requested a full tablet report, sending...
I20260808 10:02:48.833899 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25583
I20260808 10:02:48.833935 25302 ts_manager.cc:194] Registered new tserver with Master: 35a7d4add4414c549fef659f6884b2b5 (127.24.162.130:45753)
I20260808 10:02:48.834005 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:02:48.834173 25226 external_mini_cluster.cc:1428] Setting key 33bbdca5444f63f1a9965c396ff33a91
I20260808 10:02:48.834410 25302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:39993
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:02:48.861300 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:48.946669 25768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:48.946871 25768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:48.946897 25768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:48.948060 25768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:48.948105 25768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:48.948132 25768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:02:48.949512 25768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:48.949709 25768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:48.949878 25768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:48.954838 25773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:48.954917 25777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:48.955024 25768 server_base.cc:1034] running on GCE node
W20260808 10:02:48.954927 25775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:48.955348 25768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:48.955544 25768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:48.956707 25768 hybrid_clock.cc:648] HybridClock initialized: now 1786183368956703 us; error 38 us; skew 500 ppm
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:48.959154 25768 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:02:48.959591 25768 webserver.cc:460] Webserver started at http://127.24.162.131:41009/ using document root <none> and password file <none>
I20260808 10:02:48.959769 25768 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:48.959807 25768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:48.959910 25768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:48.960881 25768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "82381a4667174b03a82103ca3f50fece"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "306b74272a594cdfacef77b2c23cdda5"
server_key_iv: "7ca76aae291eb3e5917045c1bb069b86"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.961258 25768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "82381a4667174b03a82103ca3f50fece"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "306b74272a594cdfacef77b2c23cdda5"
server_key_iv: "7ca76aae291eb3e5917045c1bb069b86"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.962566 25768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:02:48.963430 25784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.963593 25768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:02:48.963737 25768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "82381a4667174b03a82103ca3f50fece"
format_stamp: "Formatted at 2026-08-08 10:02:48 on dist-test-slave-nfb5"
server_key: "306b74272a594cdfacef77b2c23cdda5"
server_key_iv: "7ca76aae291eb3e5917045c1bb069b86"
server_key_version: "encryptionkey@0"
I20260808 10:02:48.963817 25768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:48.980058 25768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:48.981906 25768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:48.982069 25768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:48.982291 25768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:48.982611 25768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:48.982668 25768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.982712 25768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:48.982749 25768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:48.989388 25768 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:34873
I20260808 10:02:48.989465 25941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:34873 every 8 connection(s)
I20260808 10:02:48.989781 25768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:02:48 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:48.996529 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25768
I20260808 10:02:48.996618 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:02:48.996789 25226 external_mini_cluster.cc:1428] Setting key 1a415e0d007366f586c55d98e816f78f
I20260808 10:02:48.999372 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.990508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:46395 (local address 127.24.162.190:38777)
0808 10:02:48.990556 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:48.990559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:48.990741 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:02:48.991275 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:48.991277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:48.991290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:48.991348 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:48.992284 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.992785 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.993785 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.993987 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.994066 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.996134 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:48.996136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:48.996136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:48.996143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:48.997867 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.998374 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.998376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.998376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.998439 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:48.998704 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:48.998705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:48.998705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:48.998828 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:48.998903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:48.999200 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:02:48.999277 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:02:48.999577 25945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:48.990462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38777 (local address 127.24.162.131:46395)
0808 10:02:48.990603 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:48.990652 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:48.991164 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:48.991413 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:48.991417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:48.991989 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:48.992173 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.992177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.992900 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.992901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.993688 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:48.993690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:48.994073 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:48.994074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:48.994213 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:48.994673 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:48.994681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:48.996029 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:48.997946 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.997953 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.997956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.998271 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.998510 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:48.998511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:48.998511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:48.998631 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:48.998905 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:48.998937 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:48.999068 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 10:02:48.999179 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:02:48.999344 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260808 10:02:49.000089 25942 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38777
I20260808 10:02:49.000167 25942 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:49.000295 25942 heartbeater.cc:511] Master 127.24.162.190:38777 requested a full tablet report, sending...
I20260808 10:02:49.000617 25301 ts_manager.cc:194] Registered new tserver with Master: 82381a4667174b03a82103ca3f50fece (127.24.162.131:34873)
I20260808 10:02:49.001021 25301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:46395
I20260808 10:02:49.009155 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:49.033613 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:49.025608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.24.162.129:38549)
0808 10:02:49.025872 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:49.025876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:49.025886 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:02:49.025912 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:49.025914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:49.025927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:49.025996 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:49.026582 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:49.027369 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:49.028054 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:49.028554 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:49.028622 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:49.030760 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:49.030766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:49.030766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:49.030772 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:49.032499 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:49.032854 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:49.032857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:49.032858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:49.032921 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:49.033093 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:49.033093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:49.033093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:49.033216 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:49.033329 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:49.033431 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:02:49.033491 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":182,"threads_started":1}
W20260808 10:02:49.034015 25452 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34364
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:49.056823 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:49.050127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.24.162.129:38549)
0808 10:02:49.050216 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:49.050218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:49.050304 (+ 86us) server_negotiation.cc:407] Connection header received
0808 10:02:49.050371 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:49.050371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:49.050380 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:49.050427 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:49.051068 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:49.051553 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:49.052245 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:49.052431 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:49.052480 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:49.054642 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:49.054644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:49.054644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:49.054650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:49.055916 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:49.056218 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:49.056219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:49.056219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:49.056270 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:49.056430 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:49.056431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:49.056431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:49.056518 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:49.056607 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:49.056671 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:02:49.056732 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:02:49.057454 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25396
I20260808 10:02:49.070055 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25583
I20260808 10:02:49.079365 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25768
I20260808 10:02:49.087906 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25256
2026-08-08T10:02:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2377 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25964](info): set up 2 sockets
Aug 08 10:02:49 dist-test-slave-nfb5 krb5kdc[25964](info): commencing operation
krb5kdc: starting...
W20260808 10:02:50.314999 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:02:50.434983 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41881
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:41881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:50.520272 25984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:50.520490 25984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:50.520514 25984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:50.521608 25984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:02:50.521646 25984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:50.521660 25984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:50.521673 25984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:02:50.521685 25984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:02:50.523109 25984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:41881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:50.523300 25984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:50.523470 25984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:50.528395 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:50.528404 25992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:50.528519 25984 server_base.cc:1034] running on GCE node
W20260808 10:02:50.528404 25994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:50.528875 25984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:50.529186 25984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:50.530339 25984 hybrid_clock.cc:648] HybridClock initialized: now 1786183370530326 us; error 41 us; skew 500 ppm
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:50.533064 25984 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:02:50.533548 25984 webserver.cc:460] Webserver started at http://127.24.162.190:35853/ using document root <none> and password file <none>
I20260808 10:02:50.533776 25984 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:50.533844 25984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:50.534003 25984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:50.535213 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "9eee8018183c4dfb9e4323ec2700b675"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "fc474fe64752415ace6c62b924e301c0"
server_key_iv: "b841184259f86173e8dacb38a08eed8d"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.535588 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "9eee8018183c4dfb9e4323ec2700b675"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "fc474fe64752415ace6c62b924e301c0"
server_key_iv: "b841184259f86173e8dacb38a08eed8d"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.537253 25984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:02:50.538125 26006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.538331 25984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:50.538452 25984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "9eee8018183c4dfb9e4323ec2700b675"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "fc474fe64752415ace6c62b924e301c0"
server_key_iv: "b841184259f86173e8dacb38a08eed8d"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.538530 25984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:50.555300 25984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:50.557750 25984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:50.557927 25984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:50.562224 25984 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:41881
I20260808 10:02:50.562259 26077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:41881 every 8 connection(s)
I20260808 10:02:50.562642 25984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:02:50.563596 26079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:50.565780 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675: Bootstrap starting.
I20260808 10:02:50.566336 26079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:50.566573 26079 log.cc:824] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:50.567276 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675: No bootstrap required, opened a new log
I20260808 10:02:50.568778 26079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } }
I20260808 10:02:50.568928 26079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:50.568953 26079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eee8018183c4dfb9e4323ec2700b675, State: Initialized, Role: FOLLOWER
I20260808 10:02:50.569087 26079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } }
I20260808 10:02:50.569168 26079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:02:50.569191 26079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:02:50.569219 26079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:50.569689 26079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } }
I20260808 10:02:50.569761 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25984
I20260808 10:02:50.569774 26079 leader_election.cc:302] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eee8018183c4dfb9e4323ec2700b675; no voters:
I20260808 10:02:50.569862 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:02:50.569986 26079 leader_election.cc:288] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:02:50.570029 26083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:50.570052 25226 external_mini_cluster.cc:1428] Setting key d66d65cc6d786b70e44648930ec92bea
I20260808 10:02:50.570243 26083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [term 1 LEADER]: Becoming Leader. State: Replica: 9eee8018183c4dfb9e4323ec2700b675, State: Running, Role: LEADER
I20260808 10:02:50.570251 26079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:02:50.570446 26083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } }
I20260808 10:02:50.570854 26085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eee8018183c4dfb9e4323ec2700b675. Latest consensus state: current_term: 1 leader_uuid: "9eee8018183c4dfb9e4323ec2700b675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } } }
I20260808 10:02:50.571000 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eee8018183c4dfb9e4323ec2700b675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eee8018183c4dfb9e4323ec2700b675" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41881 } } }
I20260808 10:02:50.571015 26085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:50.571113 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675 [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:50.571409 26099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:02:50.572027 26099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:02:50.573617 26099 catalog_manager.cc:1347] Generated new cluster ID: 95486f71608d4e3e95b21254a1017370
I20260808 10:02:50.573674 26099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:50.581048 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:50.570683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.24.162.190:41881)
0808 10:02:50.571215 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:50.571226 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:50.571261 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:02:50.571454 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:50.571463 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:50.571532 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:50.571694 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:50.572728 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.573995 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.574745 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.575247 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.575365 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:50.577660 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:50.577663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:50.577664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:50.577675 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:50.579476 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.579856 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.579859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.579860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.579962 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.580184 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.580185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.580185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.580420 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:50.580583 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:50.580700 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 10:02:50.580841 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":75,"threads_started":1}
I20260808 10:02:50.611548 26099 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:02:50.612106 26099 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:02:50.618976 26099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9eee8018183c4dfb9e4323ec2700b675: Generated new TSK 0
I20260808 10:02:50.619174 26099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:02:50.653743 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:50.736877 26120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:50.737108 26120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:50.737150 26120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:50.738289 26120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:50.738351 26120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:50.738409 26120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:02:50.740012 26120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:50.740271 26120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:50.740501 26120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:50.746500 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:50.746510 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:50.746510 26128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:50.746876 26120 server_base.cc:1034] running on GCE node
I20260808 10:02:50.747126 26120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:50.747393 26120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:50.748566 26120 hybrid_clock.cc:648] HybridClock initialized: now 1786183370748553 us; error 30 us; skew 500 ppm
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:50.751292 26120 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:02:50.751752 26120 webserver.cc:460] Webserver started at http://127.24.162.129:35825/ using document root <none> and password file <none>
I20260808 10:02:50.751935 26120 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:50.751974 26120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:50.752064 26120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:50.753033 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "437ef6e3526a4b1696fc1bdea727b5ed"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "1f4cc459fc6e3220450f0eb10d5f7025"
server_key_iv: "a13a86d7fa85727f90eeebad481e928a"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.753346 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "437ef6e3526a4b1696fc1bdea727b5ed"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "1f4cc459fc6e3220450f0eb10d5f7025"
server_key_iv: "a13a86d7fa85727f90eeebad481e928a"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.754608 26120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:02:50.755611 26143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.755808 26120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:50.755904 26120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "437ef6e3526a4b1696fc1bdea727b5ed"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "1f4cc459fc6e3220450f0eb10d5f7025"
server_key_iv: "a13a86d7fa85727f90eeebad481e928a"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.755959 26120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:50.780905 26120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:50.783017 26120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:50.783143 26120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:50.783349 26120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:50.783677 26120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:50.783712 26120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.783735 26120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:50.783790 26120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.789475 26120 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:43667
I20260808 10:02:50.789569 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:43667 every 8 connection(s)
I20260808 10:02:50.789894 26120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:50.800173 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 26120
I20260808 10:02:50.800272 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:02:50.800432 25226 external_mini_cluster.cc:1428] Setting key 3566ee73d644180a6f25249b27755a0f
I20260808 10:02:50.802230 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:50.790671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:36501 (local address 127.24.162.190:41881)
0808 10:02:50.790801 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:50.790803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:50.790950 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:02:50.792590 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:50.792602 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:50.792615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:50.792692 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:50.793551 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.794326 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.795778 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.796154 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.796223 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:50.798939 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:50.798941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:50.798942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:50.798950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:50.800568 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.801081 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.801082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.801082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.801149 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.801483 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.801484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.801484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.801625 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:50.801710 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:50.802029 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 10:02:50.802120 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 10:02:50.802404 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:50.790617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41881 (local address 127.24.162.129:36501)
0808 10:02:50.790790 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:50.790830 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:50.792463 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:50.792762 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:50.792766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:50.793167 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:50.793467 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:50.793474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.794441 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.794442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:50.795679 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:50.795682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.796254 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.796255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:50.796400 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:50.797045 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:50.797055 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:50.798802 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:50.800682 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:50.800690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:50.800693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:50.800992 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:50.801234 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:50.801235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:50.801235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:50.801399 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:50.801704 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:50.801711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:50.801868 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 10:02:50.801980 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:02:50.802144 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260808 10:02:50.805497 26312 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41881
I20260808 10:02:50.805636 26312 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:50.805832 26312 heartbeater.cc:511] Master 127.24.162.190:41881 requested a full tablet report, sending...
I20260808 10:02:50.806429 26026 ts_manager.cc:194] Registered new tserver with Master: 437ef6e3526a4b1696fc1bdea727b5ed (127.24.162.129:43667)
I20260808 10:02:50.807364 26026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:36501
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:02:50.831761 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:50.915912 26320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:50.916141 26320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:50.916167 26320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:50.917270 26320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:50.917306 26320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:50.917332 26320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:02:50.918798 26320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:50.919068 26320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:50.919262 26320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:50.924024 26327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:50.924021 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:50.924024 26329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:50.924288 26320 server_base.cc:1034] running on GCE node
I20260808 10:02:50.924535 26320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:50.924799 26320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:50.925977 26320 hybrid_clock.cc:648] HybridClock initialized: now 1786183370925964 us; error 31 us; skew 500 ppm
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:50.928579 26320 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:02:50.929049 26320 webserver.cc:460] Webserver started at http://127.24.162.130:45807/ using document root <none> and password file <none>
I20260808 10:02:50.929286 26320 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:50.929356 26320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:50.929520 26320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:50.930542 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "87f5bda176cf4cadbb25c27f0baa23c0"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "a9ee4b49d264e86ece058159a3b11390"
server_key_iv: "c3b44a4f0e927ff9f282ae430dc1dd08"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.931047 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "87f5bda176cf4cadbb25c27f0baa23c0"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "a9ee4b49d264e86ece058159a3b11390"
server_key_iv: "c3b44a4f0e927ff9f282ae430dc1dd08"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.932370 26320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:50.933104 26342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.933338 26320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:50.933447 26320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "87f5bda176cf4cadbb25c27f0baa23c0"
format_stamp: "Formatted at 2026-08-08 10:02:50 on dist-test-slave-nfb5"
server_key: "a9ee4b49d264e86ece058159a3b11390"
server_key_iv: "c3b44a4f0e927ff9f282ae430dc1dd08"
server_key_version: "encryptionkey@0"
I20260808 10:02:50.933539 26320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:50.942397 26320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:50.944782 26320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:50.944943 26320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:50.945165 26320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:50.945477 26320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:50.945531 26320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.945572 26320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:50.945600 26320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:50.951526 26320 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:42141
I20260808 10:02:50.951575 26497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:42141 every 8 connection(s)
I20260808 10:02:50.951869 26320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:02:50.956458 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 26320
I20260808 10:02:50.956559 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:02:50.956725 25226 external_mini_cluster.cc:1428] Setting key 83c46163f84ec244e42fab73899b39ba
Aug 08 10:02:50 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:50.962193 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:50.952554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:34017 (local address 127.24.162.190:41881)
0808 10:02:50.952604 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:50.952606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:50.952822 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:02:50.953765 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:50.953766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:50.953780 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:50.953844 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:50.954447 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.955029 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.956057 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.956268 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.956344 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:50.958985 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:50.958986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:50.958986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:50.958993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:50.960756 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.961141 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.961143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.961143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.961218 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:50.961477 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:50.961478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:50.961478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:50.961633 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:50.961709 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:50.961978 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:02:50.962080 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:02:50.962219 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:50.952516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41881 (local address 127.24.162.130:34017)
0808 10:02:50.952704 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:50.952737 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:50.953641 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:50.953913 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:50.953917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:50.954166 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:50.954346 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:50.954351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.955124 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.955125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:50.955960 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:50.955963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:50.956364 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:50.956365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:50.956495 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:50.957023 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:50.957034 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:50.958862 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:50.960839 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:50.960845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:50.960846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:50.961064 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:50.961308 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:50.961309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:50.961309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:50.961411 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:50.961710 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:50.961714 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:50.961821 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 10:02:50.961901 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:02:50.962014 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260808 10:02:50.962734 26498 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41881
I20260808 10:02:50.962842 26498 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:50.963059 26498 heartbeater.cc:511] Master 127.24.162.190:41881 requested a full tablet report, sending...
I20260808 10:02:50.963485 26026 ts_manager.cc:194] Registered new tserver with Master: 87f5bda176cf4cadbb25c27f0baa23c0 (127.24.162.130:42141)
I20260808 10:02:50.963867 26026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:34017
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:02:50.985176 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:51.066395 26504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:51.066591 26504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:51.066614 26504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:51.067720 26504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:51.067756 26504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:51.067782 26504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:02:51.069180 26504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41881
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:51.069362 26504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:51.069530 26504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:51.074843 26510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:51.074956 26504 server_base.cc:1034] running on GCE node
W20260808 10:02:51.074829 26511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:51.074851 26515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:51.075224 26504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:51.075461 26504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:51.076603 26504 hybrid_clock.cc:648] HybridClock initialized: now 1786183371076593 us; error 35 us; skew 500 ppm
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:51.079186 26504 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:02:51.079622 26504 webserver.cc:460] Webserver started at http://127.24.162.131:36505/ using document root <none> and password file <none>
I20260808 10:02:51.079813 26504 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:51.079861 26504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:51.079947 26504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:51.080902 26504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "3b7336a562c149efbc739f4ece839a17"
format_stamp: "Formatted at 2026-08-08 10:02:51 on dist-test-slave-nfb5"
server_key: "648c33f185ab51774fcde735082a1cd1"
server_key_iv: "46f96277b92138174b112e72cec5249d"
server_key_version: "encryptionkey@0"
I20260808 10:02:51.081259 26504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "3b7336a562c149efbc739f4ece839a17"
format_stamp: "Formatted at 2026-08-08 10:02:51 on dist-test-slave-nfb5"
server_key: "648c33f185ab51774fcde735082a1cd1"
server_key_iv: "46f96277b92138174b112e72cec5249d"
server_key_version: "encryptionkey@0"
I20260808 10:02:51.082566 26504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:02:51.083360 26525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:51.083535 26504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:51.083638 26504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "3b7336a562c149efbc739f4ece839a17"
format_stamp: "Formatted at 2026-08-08 10:02:51 on dist-test-slave-nfb5"
server_key: "648c33f185ab51774fcde735082a1cd1"
server_key_iv: "46f96277b92138174b112e72cec5249d"
server_key_version: "encryptionkey@0"
I20260808 10:02:51.083719 26504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:51.092986 26504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:51.095346 26504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:51.095506 26504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:51.095723 26504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:02:51.096040 26504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:51.096097 26504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:51.096138 26504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:51.096165 26504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:51.101955 26504 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:42625
I20260808 10:02:51.102042 26690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:42625 every 8 connection(s)
I20260808 10:02:51.102314 26504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:51.110141 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 26504
I20260808 10:02:51.110242 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:02:51.110433 25226 external_mini_cluster.cc:1428] Setting key 4ea619dbaf817b5d65e7cd1f220036fb
I20260808 10:02:51.111557 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.103182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:35673 (local address 127.24.162.190:41881)
0808 10:02:51.103239 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.103242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.103451 (+ 209us) server_negotiation.cc:407] Connection header received
0808 10:02:51.104292 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.104293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.104305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.104369 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:51.104911 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.105448 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.106382 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.106581 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.106644 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.108659 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:51.108660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:51.108660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:51.108667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:51.109945 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.110433 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.110434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.110435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.110491 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.110759 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.110760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.110761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.110921 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:51.111020 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.111360 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.111451 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:02:51.111685 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.103159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41881 (local address 127.24.162.131:35673)
0808 10:02:51.103323 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.103356 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.104198 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.104418 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.104422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.104656 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:51.104837 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.104840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.105529 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.105529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.106300 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.106301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.106654 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.106655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.106756 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.107270 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:51.107278 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:51.108572 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:51.110031 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:51.110039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:51.110041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:51.110357 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:51.110594 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:51.110595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:51.110595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:51.110702 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:51.111014 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:51.111021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:51.111188 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.111289 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.111436 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260808 10:02:51.112192 26691 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41881
I20260808 10:02:51.112294 26691 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:51.112465 26691 heartbeater.cc:511] Master 127.24.162.190:41881 requested a full tablet report, sending...
I20260808 10:02:51.112896 26026 ts_manager.cc:194] Registered new tserver with Master: 3b7336a562c149efbc739f4ece839a17 (127.24.162.131:42625)
I20260808 10:02:51.113284 26026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:35673
I20260808 10:02:51.122336 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:02:51.130136 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.124364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.24.162.190:41881)
0808 10:02:51.124415 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.124417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.124515 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:02:51.124551 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.124551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.124560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.124613 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:51.125182 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.125653 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.126396 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.126597 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.126711 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.127888 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:51.127889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:51.127889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:51.127896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:51.129052 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.129417 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.129418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.129418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.129466 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.129631 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.129631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.129631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.129799 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:51.129876 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.129980 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.130037 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:02:51.132714 26026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:02:51.134477 26026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:51.141841 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.137759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.24.162.131:42625)
0808 10:02:51.138124 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.138127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.138135 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:02:51.138174 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.138175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.138206 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.138300 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:51.138489 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139405 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.140910 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.141397 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.141487 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.141545 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.141703 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.141749 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":72,"threads_started":1}
I20260808 10:02:51.141832 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.137634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:43667 (local address 127.0.0.1:47850)
0808 10:02:51.137837 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.137879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.137973 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.138500 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.138501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.138504 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:51.138568 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.138570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139917 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.139918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.140732 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.140736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.141517 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.141518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.141584 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.141605 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.141668 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.141735 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260808 10:02:51.141836 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.137664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:42625 (local address 127.0.0.1:36730)
0808 10:02:51.137938 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.137950 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.138008 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.138317 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.138321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.138323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:51.138403 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.138405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139516 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.139518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.140799 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.140802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.141497 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.141498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.141582 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.141595 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.141671 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.141744 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":26,"thread_start_us":95,"threads_started":1}
I20260808 10:02:51.142433 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.137745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.24.162.129:43667)
0808 10:02:51.138326 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.138331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.138340 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:51.138379 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.138381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.138396 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.138698 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:51.138787 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139810 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.140834 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.141408 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.141502 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.141559 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.142298 (+ 739us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.142356 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":97,"threads_started":1}
I20260808 10:02:51.142663 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.137846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:42141 (local address 127.0.0.1:44610)
0808 10:02:51.137978 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.137988 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.138032 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.138412 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.138413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.138415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:51.138451 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.138452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139716 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.139717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.140799 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.140802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.142386 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.142387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.142450 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.142454 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.142520 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.142594 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260808 10:02:51.143093 26595 tablet_service.cc:1467] Processing CreateTablet for tablet c191d77ac368439481a4c237149d9c4a (DEFAULT_TABLE table=test-table [id=52e08c8e18a44e2ab1d807970c9289d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:51.143205 26214 tablet_service.cc:1467] Processing CreateTablet for tablet c191d77ac368439481a4c237149d9c4a (DEFAULT_TABLE table=test-table [id=52e08c8e18a44e2ab1d807970c9289d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:51.143502 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c191d77ac368439481a4c237149d9c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:51.143535 26214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c191d77ac368439481a4c237149d9c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:51.146028 26715 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Bootstrap starting.
I20260808 10:02:51.146409 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.138069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.24.162.130:42141)
0808 10:02:51.138251 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.138256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.138269 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:02:51.138308 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.138310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.138326 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.138419 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:51.138500 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.139608 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.141582 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.142290 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.142374 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.142423 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.146295 (+ 3872us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.146349 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260808 10:02:51.146652 26717 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: Bootstrap starting.
I20260808 10:02:51.146725 26715 tablet_bootstrap.cc:654] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:51.147095 26406 tablet_service.cc:1467] Processing CreateTablet for tablet c191d77ac368439481a4c237149d9c4a (DEFAULT_TABLE table=test-table [id=52e08c8e18a44e2ab1d807970c9289d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:51.147318 26717 tablet_bootstrap.cc:654] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:51.147444 26406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c191d77ac368439481a4c237149d9c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:51.147485 26715 log.cc:824] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:51.148070 26717 log.cc:824] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:51.148288 26715 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: No bootstrap required, opened a new log
I20260808 10:02:51.148357 26715 ts_tablet_manager.cc:1397] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:02:51.148808 26717 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: No bootstrap required, opened a new log
I20260808 10:02:51.148882 26717 ts_tablet_manager.cc:1397] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:02:51.150056 26715 raft_consensus.cc:348] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.150223 26715 raft_consensus.cc:374] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:51.150267 26715 raft_consensus.cc:729] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437ef6e3526a4b1696fc1bdea727b5ed, State: Initialized, Role: FOLLOWER
I20260808 10:02:51.150403 26715 consensus_queue.cc:260] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.150753 26715 ts_tablet_manager.cc:1428] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:02:51.151013 26725 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Bootstrap starting.
I20260808 10:02:51.150965 26717 raft_consensus.cc:348] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.151157 26717 raft_consensus.cc:374] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:51.151204 26717 raft_consensus.cc:729] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7336a562c149efbc739f4ece839a17, State: Initialized, Role: FOLLOWER
I20260808 10:02:51.151350 26717 consensus_queue.cc:260] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.151640 26717 ts_tablet_manager.cc:1428] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:02:51.151683 26691 heartbeater.cc:503] Master 127.24.162.190:41881 was elected leader, sending a full tablet report...
I20260808 10:02:51.151705 26725 tablet_bootstrap.cc:654] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:51.151751 26312 heartbeater.cc:503] Master 127.24.162.190:41881 was elected leader, sending a full tablet report...
I20260808 10:02:51.152345 26725 log.cc:824] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:51.153122 26725 tablet_bootstrap.cc:492] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: No bootstrap required, opened a new log
I20260808 10:02:51.153204 26725 ts_tablet_manager.cc:1397] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:02:51.154788 26725 raft_consensus.cc:348] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.154968 26725 raft_consensus.cc:374] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:51.155027 26725 raft_consensus.cc:729] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f5bda176cf4cadbb25c27f0baa23c0, State: Initialized, Role: FOLLOWER
I20260808 10:02:51.155141 26725 consensus_queue.cc:260] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.155448 26725 ts_tablet_manager.cc:1428] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:02:51.155468 26498 heartbeater.cc:503] Master 127.24.162.190:41881 was elected leader, sending a full tablet report...
W20260808 10:02:51.202505 26499 tablet.cc:2369] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:02:51.205879 26728 raft_consensus.cc:482] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:51.205973 26728 raft_consensus.cc:504] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.206614 26728 leader_election.cc:288] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 437ef6e3526a4b1696fc1bdea727b5ed (127.24.162.129:43667), 3b7336a562c149efbc739f4ece839a17 (127.24.162.131:42625)
I20260808 10:02:51.209518 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.206828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:42625 (local address 127.24.162.130:34351)
0808 10:02:51.206883 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.206895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.206981 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.207134 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.207135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.207136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:51.207215 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.207217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.207961 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.207962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.208688 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.208690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.209244 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.209245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.209338 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.209341 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.209409 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.209451 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:02:51.209518 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.206906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:34351 (local address 127.24.162.131:42625)
0808 10:02:51.206969 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.206971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.206981 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:02:51.207041 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.207042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.207051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.207128 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:51.207291 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.207877 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.208783 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.209160 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.209248 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.209256 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.209415 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.209452 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:02:51.209793 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.206959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:51833 (local address 127.24.162.129:43667)
0808 10:02:51.207011 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.207014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.207181 (+ 167us) server_negotiation.cc:407] Connection header received
0808 10:02:51.207296 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.207297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.207308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.207384 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:51.207542 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.208067 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.209006 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.209379 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.209463 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.209471 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.209663 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.209714 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:02:51.209913 26622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c191d77ac368439481a4c237149d9c4a" candidate_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7336a562c149efbc739f4ece839a17" is_pre_election: true
I20260808 10:02:51.210014 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.206875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:43667 (local address 127.24.162.130:51833)
0808 10:02:51.207099 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:51.207108 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:51.207212 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:51.207389 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:51.207390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:51.207392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:51.207470 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.207472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.208156 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.208157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.208911 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:51.208913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.209474 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.209475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:51.209551 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.209554 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:02:51.209900 (+ 346us) client_negotiation.cc:240] Negotiation successful
0808 10:02:51.209942 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":164,"threads_started":1}
I20260808 10:02:51.210131 26622 raft_consensus.cc:2454] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f5bda176cf4cadbb25c27f0baa23c0 in term 0.
I20260808 10:02:51.210444 26347 leader_election.cc:302] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7336a562c149efbc739f4ece839a17, 87f5bda176cf4cadbb25c27f0baa23c0; no voters:
I20260808 10:02:51.210443 26247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c191d77ac368439481a4c237149d9c4a" candidate_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" is_pre_election: true
I20260808 10:02:51.210584 26247 raft_consensus.cc:2454] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f5bda176cf4cadbb25c27f0baa23c0 in term 0.
I20260808 10:02:51.210597 26728 raft_consensus.cc:2781] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:51.210661 26728 raft_consensus.cc:482] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:51.210685 26728 raft_consensus.cc:3037] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:51.211411 26728 raft_consensus.cc:504] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.211549 26728 leader_election.cc:288] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [CANDIDATE]: Term 1 election: Requested vote from peers 437ef6e3526a4b1696fc1bdea727b5ed (127.24.162.129:43667), 3b7336a562c149efbc739f4ece839a17 (127.24.162.131:42625)
I20260808 10:02:51.211735 26622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c191d77ac368439481a4c237149d9c4a" candidate_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7336a562c149efbc739f4ece839a17"
I20260808 10:02:51.211719 26247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c191d77ac368439481a4c237149d9c4a" candidate_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437ef6e3526a4b1696fc1bdea727b5ed"
I20260808 10:02:51.211809 26622 raft_consensus.cc:3037] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:51.211812 26247 raft_consensus.cc:3037] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:51.212718 26622 raft_consensus.cc:2454] T c191d77ac368439481a4c237149d9c4a P 3b7336a562c149efbc739f4ece839a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f5bda176cf4cadbb25c27f0baa23c0 in term 1.
I20260808 10:02:51.212718 26247 raft_consensus.cc:2454] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f5bda176cf4cadbb25c27f0baa23c0 in term 1.
I20260808 10:02:51.212955 26345 leader_election.cc:302] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437ef6e3526a4b1696fc1bdea727b5ed, 87f5bda176cf4cadbb25c27f0baa23c0; no voters:
I20260808 10:02:51.213075 26728 raft_consensus.cc:2781] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:51.213310 26728 raft_consensus.cc:686] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [term 1 LEADER]: Becoming Leader. State: Replica: 87f5bda176cf4cadbb25c27f0baa23c0, State: Running, Role: LEADER
I20260808 10:02:51.213433 26728 consensus_queue.cc:237] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } }
I20260808 10:02:51.214449 26026 catalog_manager.cc:5495] T c191d77ac368439481a4c237149d9c4a P 87f5bda176cf4cadbb25c27f0baa23c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87f5bda176cf4cadbb25c27f0baa23c0 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "437ef6e3526a4b1696fc1bdea727b5ed" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7336a562c149efbc739f4ece839a17" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87f5bda176cf4cadbb25c27f0baa23c0" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42141 } health_report { overall_health: HEALTHY } } }
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:51.239495 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.232008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.24.162.129:43667)
0808 10:02:51.232073 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.232075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.232161 (+ 86us) server_negotiation.cc:407] Connection header received
0808 10:02:51.232214 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.232215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.232224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.232276 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:51.232892 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.233364 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.234131 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.234319 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.234379 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.236796 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:51.236806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:51.236806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:51.236813 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:51.238405 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.238733 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.238736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.238737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.238798 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.239032 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.239033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.239033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.239185 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:51.239299 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.239358 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.239424 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:51.265345 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.258131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.24.162.129:43667)
0808 10:02:51.258219 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.258222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.258279 (+ 57us) server_negotiation.cc:407] Connection header received
0808 10:02:51.258326 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.258327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.258336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.258388 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:51.259013 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.259488 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.260230 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.260435 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.260504 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.262765 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:51.262767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:51.262767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:51.262774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:51.264170 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.264542 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.264544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.264544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.264608 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.264828 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.264829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.264829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.264922 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:51.265002 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.265144 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.265210 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:51.289978 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:51.282353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.24.162.129:43667)
0808 10:02:51.282458 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:51.282461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:51.282563 (+ 102us) server_negotiation.cc:407] Connection header received
0808 10:02:51.282641 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:51.282643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:51.282653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:51.282730 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:51.283426 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.283920 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.284669 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:51.284862 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:51.284941 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:51.287365 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:51.287367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:51.287368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:51.287375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:51.288762 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.289326 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.289328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.289328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.289386 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:51.289573 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:51.289574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:51.289574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:51.289670 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:51.289751 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:51.289850 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 10:02:51.289910 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 10:02:51.290658 26314 tablet.cc:2369] T c191d77ac368439481a4c237149d9c4a P 437ef6e3526a4b1696fc1bdea727b5ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:02:51.290798 26182 tablet_service.cc:2908] Rejecting scan request for tablet c191d77ac368439481a4c237149d9c4a: Uninitialized: clean time has not yet been initialized
I20260808 10:02:51.292165 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 26120
I20260808 10:02:51.307108 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 26320
I20260808 10:02:51.315541 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 26504
I20260808 10:02:51.324167 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25984
2026-08-08T10:02:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2236 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[26749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[26749](info): set up 2 sockets
Aug 08 10:02:51 dist-test-slave-nfb5 krb5kdc[26749](info): commencing operation
krb5kdc: starting...
W20260808 10:02:52.568969 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:02:52 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:02:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:02:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:02:52.691298 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44213
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:44213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:52.773068 26766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:52.773268 26766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:52.773291 26766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:52.774441 26766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:02:52.774483 26766 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:02:52.774497 26766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:52.774510 26766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:52.774570 26766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:02:52.774587 26766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:02:52.776065 26766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:44213
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:52.776324 26766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:52.776561 26766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:52.781420 26772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:52.781376 26775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:52.781505 26766 server_base.cc:1034] running on GCE node
W20260808 10:02:52.781620 26777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:52.781869 26766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:52.782111 26766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:52.783254 26766 hybrid_clock.cc:648] HybridClock initialized: now 1786183372783224 us; error 44 us; skew 500 ppm
Aug 08 10:02:52 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183372, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:52.789340 26766 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:02:52.789812 26766 webserver.cc:460] Webserver started at http://127.24.162.190:35305/ using document root <none> and password file <none>
I20260808 10:02:52.790042 26766 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:52.790112 26766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:52.790268 26766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:52.791666 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "6363d74eac334595a749c6815f798cce"
format_stamp: "Formatted at 2026-08-08 10:02:52 on dist-test-slave-nfb5"
server_key: "fb9f6a41b390fb6348cdfc8a45cf9ced"
server_key_iv: "c31f28dcc8cbec489db05e01219d3943"
server_key_version: "encryptionkey@0"
I20260808 10:02:52.792130 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "6363d74eac334595a749c6815f798cce"
format_stamp: "Formatted at 2026-08-08 10:02:52 on dist-test-slave-nfb5"
server_key: "fb9f6a41b390fb6348cdfc8a45cf9ced"
server_key_iv: "c31f28dcc8cbec489db05e01219d3943"
server_key_version: "encryptionkey@0"
I20260808 10:02:52.794219 26766 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:02:52.795238 26788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:52.795416 26766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:02:52.795504 26766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "6363d74eac334595a749c6815f798cce"
format_stamp: "Formatted at 2026-08-08 10:02:52 on dist-test-slave-nfb5"
server_key: "fb9f6a41b390fb6348cdfc8a45cf9ced"
server_key_iv: "c31f28dcc8cbec489db05e01219d3943"
server_key_version: "encryptionkey@0"
I20260808 10:02:52.795567 26766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:52.832911 26766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:52.835013 26766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:52.835175 26766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:52.840257 26766 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:44213
I20260808 10:02:52.840303 26877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:44213 every 8 connection(s)
I20260808 10:02:52.840631 26766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:02:52.841280 26881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:52.843593 26881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Bootstrap starting.
I20260808 10:02:52.844187 26881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:52.844494 26881 log.cc:824] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:52.845230 26881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: No bootstrap required, opened a new log
I20260808 10:02:52.846580 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 26766
I20260808 10:02:52.846659 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:02:52.846834 25226 external_mini_cluster.cc:1428] Setting key d1b5406b99bad14962e7d6a06fe5b6c7
I20260808 10:02:52.847846 26881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } }
I20260808 10:02:52.848002 26881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:52.848068 26881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6363d74eac334595a749c6815f798cce, State: Initialized, Role: FOLLOWER
I20260808 10:02:52.848248 26881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } }
I20260808 10:02:52.848351 26881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:02:52.848398 26881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:02:52.848452 26881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:52.848984 26881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } }
I20260808 10:02:52.849107 26881 leader_election.cc:302] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6363d74eac334595a749c6815f798cce; no voters:
I20260808 10:02:52.849279 26881 leader_election.cc:288] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:02:52.849328 26891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:52.849499 26891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [term 1 LEADER]: Becoming Leader. State: Replica: 6363d74eac334595a749c6815f798cce, State: Running, Role: LEADER
I20260808 10:02:52.849562 26881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:02:52.849675 26891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } }
I20260808 10:02:52.850073 26891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6363d74eac334595a749c6815f798cce. Latest consensus state: current_term: 1 leader_uuid: "6363d74eac334595a749c6815f798cce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } } }
I20260808 10:02:52.850346 26891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:52.850265 26892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6363d74eac334595a749c6815f798cce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363d74eac334595a749c6815f798cce" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44213 } } }
I20260808 10:02:52.850761 26892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:52.850963 26903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:02:52.851508 26903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:02:52.853106 26903 catalog_manager.cc:1347] Generated new cluster ID: 16af73b7241c4c14ae25fb01801e253c
I20260808 10:02:52.853168 26903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:02:52 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:52.859337 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:52.847386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.24.162.190:44213)
0808 10:02:52.847656 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:52.847664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:52.847689 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:02:52.847805 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:52.847810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:52.847853 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:52.847993 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:52.848746 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:52.850073 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:52.851301 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:52.851772 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:52.851900 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:52.854617 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:52.854621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:52.854621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:52.854633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:52.857404 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:52.857771 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:52.857773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:52.857774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:52.857894 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:52.858065 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:52.858065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:52.858065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:52.858346 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:52.858483 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:52.858981 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 10:02:52.859116 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":166,"threads_started":1}
I20260808 10:02:52.879937 26903 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:02:52.880493 26903 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:02:52.883924 26903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Generated new TSK 0
I20260808 10:02:52.884114 26903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:02:52.917958 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44213
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:53.007854 26920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:53.008051 26920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:53.008075 26920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:53.009205 26920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:53.009248 26920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:53.009268 26920 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:53.009321 26920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:02:53.010772 26920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:44213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:53.011040 26920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:53.011270 26920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:53.016182 26929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:53.016247 26920 server_base.cc:1034] running on GCE node
W20260808 10:02:53.016157 26931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:53.016259 26928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:53.016506 26920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:53.016746 26920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:53.017923 26920 hybrid_clock.cc:648] HybridClock initialized: now 1786183373017909 us; error 31 us; skew 500 ppm
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:53.020609 26920 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:02:53.021070 26920 webserver.cc:460] Webserver started at http://127.24.162.129:42043/ using document root <none> and password file <none>
I20260808 10:02:53.021247 26920 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:53.021288 26920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:53.021384 26920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:53.022349 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "66fffac3e5a74ab285b3737c3b75b572"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "9a21dd8ba090880dcbaf16b7571053cc"
server_key_iv: "22237c4d40bd4631c9da6073dfa203e1"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.022673 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "66fffac3e5a74ab285b3737c3b75b572"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "9a21dd8ba090880dcbaf16b7571053cc"
server_key_iv: "22237c4d40bd4631c9da6073dfa203e1"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.024016 26920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:02:53.024696 26942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.024863 26920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:53.024947 26920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "66fffac3e5a74ab285b3737c3b75b572"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "9a21dd8ba090880dcbaf16b7571053cc"
server_key_iv: "22237c4d40bd4631c9da6073dfa203e1"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.025000 26920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:53.042851 26920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:53.045204 26920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:53.045324 26920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:53.045775 26920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:53.045812 26920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.045835 26920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:53.045849 26920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.052853 26920 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:37347
I20260808 10:02:53.052908 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:37347 every 8 connection(s)
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:53.053211 26920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:02:53.053625 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 26920
I20260808 10:02:53.053725 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:02:53.053907 25226 external_mini_cluster.cc:1428] Setting key b00bf7a18abaa227e1853c9d7d3a79e6
I20260808 10:02:53.057020 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.046371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.129:55085)
0808 10:02:53.046560 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.046598 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.047139 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.047418 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.047423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.047794 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.048071 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.048076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.049146 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.049146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.050552 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.050555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.051164 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.051165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.051291 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.051833 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.051845 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.053656 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.055525 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.055533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.055535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.055864 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.056071 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.056072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.056072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.056217 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.056536 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.056542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.056657 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.056744 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.056867 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260808 10:02:53.057014 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.046453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:55085 (local address 127.24.162.190:44213)
0808 10:02:53.046527 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.046530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.046690 (+ 160us) server_negotiation.cc:407] Connection header received
0808 10:02:53.047249 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.047250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.047266 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.047324 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.048189 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.049038 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.050648 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.051057 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.051138 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.053778 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.053780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.053780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.053791 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.055445 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.055950 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.055951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.055951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.056004 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.056295 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.056297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.056297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.056453 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.056534 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.056792 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.056889 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:02:53.057799 27121 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44213
I20260808 10:02:53.057904 27121 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:53.058092 27121 heartbeater.cc:511] Master 127.24.162.190:44213 requested a full tablet report, sending...
I20260808 10:02:53.058609 26810 ts_manager.cc:194] Registered new tserver with Master: 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129:37347)
I20260808 10:02:53.059587 26810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:55085
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
I20260808 10:02:53.065207 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.058826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.129:34879)
0808 10:02:53.059003 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.059016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.059083 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.059222 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.059222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.059417 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.059471 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.059472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.060459 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.060460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.061405 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.061408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.061764 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.061765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.061851 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.062259 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.062265 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.062676 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.064364 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.064365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.064365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.064526 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.064700 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.064701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.064701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.064753 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.064936 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.064936 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.064971 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.065058 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.065141 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260808 10:02:53.065332 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.058746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:34879 (local address 127.24.162.190:44213)
0808 10:02:53.058785 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.058787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.059096 (+ 309us) server_negotiation.cc:407] Connection header received
0808 10:02:53.059140 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.059141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.059153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.059212 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.059536 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.060327 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.061500 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.061681 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.061740 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.062788 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.062789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.062789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.062795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.064284 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.064591 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.064592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.064593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.064641 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.064800 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.064800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.064800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.064887 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.064932 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.065117 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.065219 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:02:53.081724 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44213
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:53.164090 27132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:53.164291 27132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:53.164322 27132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:53.165388 27132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:53.165422 27132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:53.165442 27132 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:53.165460 27132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:02:53.166946 27132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:44213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:53.167147 27132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:53.167338 27132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:53.172276 27138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:53.172356 27139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:53.172282 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:53.172523 27132 server_base.cc:1034] running on GCE node
I20260808 10:02:53.172770 27132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:53.173010 27132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:53.174153 27132 hybrid_clock.cc:648] HybridClock initialized: now 1786183373174136 us; error 36 us; skew 500 ppm
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:53.176882 27132 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:02:53.177309 27132 webserver.cc:460] Webserver started at http://127.24.162.130:34295/ using document root <none> and password file <none>
I20260808 10:02:53.177559 27132 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:53.177628 27132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:53.177784 27132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:53.178815 27132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "0b0a816337d64108a6ca7abac318cd5c"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "fb333dd17cba0fde3cb01237c7d27639"
server_key_iv: "17e6fa8c08616d6485319861404db37b"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.179236 27132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0a816337d64108a6ca7abac318cd5c"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "fb333dd17cba0fde3cb01237c7d27639"
server_key_iv: "17e6fa8c08616d6485319861404db37b"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.180585 27132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:53.181389 27155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.181562 27132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:53.181674 27132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "0b0a816337d64108a6ca7abac318cd5c"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "fb333dd17cba0fde3cb01237c7d27639"
server_key_iv: "17e6fa8c08616d6485319861404db37b"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.181787 27132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:53.191665 27132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:53.194149 27132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:53.194300 27132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:53.194978 27132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:53.195050 27132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.195091 27132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:53.195114 27132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:53.202674 27132 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:42069
I20260808 10:02:53.203087 27132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:02:53.202747 27324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:42069 every 8 connection(s)
I20260808 10:02:53.205439 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.195385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:53153 (local address 127.24.162.190:44213)
0808 10:02:53.195461 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.195463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.195686 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:02:53.196406 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.196407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.196417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.196463 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.197352 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.198150 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.199946 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.200157 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.200238 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.202828 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.202829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.202829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.202845 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.204357 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.204721 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.204722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.204722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.204775 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.204951 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.204952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.204952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.205049 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.205099 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.205319 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.205377 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:02:53.205610 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.195362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.130:53153)
0808 10:02:53.195532 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.195581 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.196292 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.196540 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.196544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.196934 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.197242 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.197248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.198251 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.198252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.199831 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.199835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.200274 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.200275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.200444 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.200979 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.200987 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.202729 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.204432 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.204438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.204440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.204652 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.204816 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.204817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.204817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.204902 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.205107 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.205111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.205218 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.205299 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.205423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260808 10:02:53.206108 27326 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44213
I20260808 10:02:53.206192 27326 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:53.206372 27326 heartbeater.cc:511] Master 127.24.162.190:44213 requested a full tablet report, sending...
I20260808 10:02:53.206761 26810 ts_manager.cc:194] Registered new tserver with Master: 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069)
I20260808 10:02:53.206851 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 27132
I20260808 10:02:53.206956 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:02:53.207134 25226 external_mini_cluster.cc:1428] Setting key d11917fb569025f4169a381dedf85c13
I20260808 10:02:53.207284 26810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:53153
I20260808 10:02:53.212939 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.207292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:39105 (local address 127.24.162.190:44213)
0808 10:02:53.207421 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.207423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.207621 (+ 198us) server_negotiation.cc:407] Connection header received
0808 10:02:53.207652 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.207653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.207663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.207719 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.208112 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.208562 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.209264 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.209467 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.209540 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.210382 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.210383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.210384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.210391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.211698 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.212050 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.212051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.212052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.212130 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.212389 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.212389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.212390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.212522 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.212578 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.212801 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.212882 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260808 10:02:53.213027 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.207209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.130:39105)
0808 10:02:53.207532 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.207544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.207611 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.207731 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.207732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.207950 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.208024 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.208026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.208657 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.208658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.209161 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.209163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.209539 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.209540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.209614 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.210033 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.210037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.210290 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.211793 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.211794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.211794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.211976 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.212228 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.212230 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.212230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.212312 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.212617 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.212618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.212670 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.212757 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.212869 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:02:53.235162 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44213
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:53.317549 27337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:53.317766 27337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:53.317791 27337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:53.318886 27337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:53.318979 27337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:53.319023 27337 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:53.319087 27337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:02:53.320761 27337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:44213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:53.321000 27337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:53.321260 27337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:53.326231 27343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:53.326242 27344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:53.326303 27346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:53.326534 27337 server_base.cc:1034] running on GCE node
I20260808 10:02:53.326763 27337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:53.327019 27337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:53.328164 27337 hybrid_clock.cc:648] HybridClock initialized: now 1786183373328132 us; error 48 us; skew 500 ppm
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:53.330890 27337 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:02:53.331430 27337 webserver.cc:460] Webserver started at http://127.24.162.131:37703/ using document root <none> and password file <none>
I20260808 10:02:53.331665 27337 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:53.331734 27337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:53.331898 27337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:53.332928 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "d5059dcac2eb4fd2982857ca7fa3a523"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "cca251d093f61d94d2473c349bfe9291"
server_key_iv: "22bf9b3722fba97a0d4dbdd21ad42961"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.333302 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "d5059dcac2eb4fd2982857ca7fa3a523"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "cca251d093f61d94d2473c349bfe9291"
server_key_iv: "22bf9b3722fba97a0d4dbdd21ad42961"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.334713 27337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:02:53.335542 27356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.335758 27337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:02:53.335865 27337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "d5059dcac2eb4fd2982857ca7fa3a523"
format_stamp: "Formatted at 2026-08-08 10:02:53 on dist-test-slave-nfb5"
server_key: "cca251d093f61d94d2473c349bfe9291"
server_key_iv: "22bf9b3722fba97a0d4dbdd21ad42961"
server_key_version: "encryptionkey@0"
I20260808 10:02:53.335953 27337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:53.358419 27337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:53.360563 27337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:53.360787 27337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:53.361363 27337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:53.361423 27337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.361469 27337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:53.361487 27337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:53.368937 27337 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:40591
I20260808 10:02:53.369062 27521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:40591 every 8 connection(s)
I20260808 10:02:53.369359 27337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:02:53.369966 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 27337
I20260808 10:02:53.370060 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:02:53.370224 25226 external_mini_cluster.cc:1428] Setting key e6887bfab9dc37bef86d161eb1d4b8bb
I20260808 10:02:53.370812 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.361822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:58735 (local address 127.24.162.190:44213)
0808 10:02:53.361897 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.361900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.362091 (+ 191us) server_negotiation.cc:407] Connection header received
0808 10:02:53.362553 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.362554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.362569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.362658 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.363333 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.363837 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.364878 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.365079 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.365159 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.367422 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.367424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.367424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.367432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.369245 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.369753 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.369755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.369755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.369844 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.370085 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.370086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.370087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.370222 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.370299 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.370614 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.370712 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:02:53.370975 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.361800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.131:58735)
0808 10:02:53.361956 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.361996 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.362461 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.362705 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.362708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.363043 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.363225 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.363230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.363952 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.363953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.364740 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.364741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.365165 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.365167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.365288 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.365810 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.365825 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.367314 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.369341 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.369345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.369347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.369658 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.369909 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.369909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.369910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.370016 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.370302 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.370309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.370463 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.370569 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.370735 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260808 10:02:53.371417 27523 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44213
I20260808 10:02:53.371524 27523 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:53.371709 27523 heartbeater.cc:511] Master 127.24.162.190:44213 requested a full tablet report, sending...
I20260808 10:02:53.372105 26810 ts_manager.cc:194] Registered new tserver with Master: d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591)
I20260808 10:02:53.372545 26810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:58735
I20260808 10:02:53.377770 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.372477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:56229 (local address 127.24.162.190:44213)
0808 10:02:53.372515 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.372516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.372602 (+ 86us) server_negotiation.cc:407] Connection header received
0808 10:02:53.372661 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.372661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.372670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.372735 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.373046 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.373508 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.374193 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.374399 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.374473 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.375372 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.375373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.375373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.375380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.376635 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.376977 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.376978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.376978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.377033 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.377240 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.377241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.377241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.377354 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.377426 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.377612 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.377681 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:02:53.377779 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.372412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.24.162.131:56229)
0808 10:02:53.372523 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.372532 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.372592 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.372738 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.372739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.372919 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.372966 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.372967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.373607 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.373608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.374109 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.374110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.374472 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.374473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.374546 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.374977 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.374982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.375254 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.376730 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.376731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.376732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.376905 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.377109 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.377110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.377110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.377168 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.377429 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.377431 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.377474 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.377583 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.377682 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
I20260808 10:02:53.381887 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:53.405742 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.398621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.24.162.190:44213)
0808 10:02:53.398754 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.398756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.398764 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:02:53.398803 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.398803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.398815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.398870 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.399483 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.400040 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.400785 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.400997 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.401074 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.403360 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.403361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.403362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.403368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.404761 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.405078 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.405079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.405079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.405129 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.405326 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.405326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.405326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.405426 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.405502 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.405597 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.405654 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 10:02:53.407516 26810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:02:53.409051 26810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:53.415802 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37347 (local address 127.0.0.1:59240)
0808 10:02:53.411494 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.411535 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.411602 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.411931 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.411934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.411937 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.412237 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.412239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.413186 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.413187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.414447 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.414450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.415422 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.415423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.415513 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.415530 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.415603 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.415713 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":58,"thread_start_us":57,"threads_started":1}
I20260808 10:02:53.415802 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.24.162.129:37347)
0808 10:02:53.411775 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.411778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.411787 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:53.411817 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.411818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.411830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.411988 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.412321 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.413099 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.414536 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.415328 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.415416 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.415480 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.415635 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.415684 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260808 10:02:53.415990 27547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:40591 (local address 127.0.0.1:43056)
0808 10:02:53.411834 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.411843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.411895 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.412081 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.412082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.412084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.412151 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.412153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.413468 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.413469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.414559 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.414561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.415461 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.415462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.415544 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.415548 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.415575 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.415684 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":94,"threads_started":1}
I20260808 10:02:53.416064 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.24.162.131:40591)
0808 10:02:53.411775 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.411780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.411892 (+ 112us) server_negotiation.cc:407] Connection header received
0808 10:02:53.411960 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.411963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.411980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.412229 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.412287 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.413377 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.414644 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.415363 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.415797 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.415871 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.415899 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.415951 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260808 10:02:53.416610 27430 tablet_service.cc:1467] Processing CreateTablet for tablet afc8fea467d3489eaf571a78858d0560 (DEFAULT_TABLE table=test-table [id=c2af354104d54fc88ad4b0543438fcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.416656 27023 tablet_service.cc:1467] Processing CreateTablet for tablet afc8fea467d3489eaf571a78858d0560 (DEFAULT_TABLE table=test-table [id=c2af354104d54fc88ad4b0543438fcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.417007 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc8fea467d3489eaf571a78858d0560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.417021 27023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc8fea467d3489eaf571a78858d0560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.417102 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:42069 (local address 127.0.0.1:41194)
0808 10:02:53.411730 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.411743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.411812 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.412170 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.412171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.412172 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.412212 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.412214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.414641 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.414641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.415416 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.415418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.416752 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.416753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.416877 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.416881 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.416943 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.416981 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260808 10:02:53.417325 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.411544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.24.162.130:42069)
0808 10:02:53.412031 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.412036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.412045 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:53.412081 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.412083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.412096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.412175 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.412282 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.413381 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.415526 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.416649 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.416734 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.416798 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.416957 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.417000 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":95,"threads_started":1}
I20260808 10:02:53.417934 27230 tablet_service.cc:1467] Processing CreateTablet for tablet afc8fea467d3489eaf571a78858d0560 (DEFAULT_TABLE table=test-table [id=c2af354104d54fc88ad4b0543438fcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.418259 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc8fea467d3489eaf571a78858d0560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.419759 27552 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Bootstrap starting.
I20260808 10:02:53.420713 27552 tablet_bootstrap.cc:654] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.421036 27552 log.cc:824] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:53.421823 27552 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: No bootstrap required, opened a new log
I20260808 10:02:53.421902 27552 ts_tablet_manager.cc:1397] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:02:53.422302 27555 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Bootstrap starting.
I20260808 10:02:53.422435 27553 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Bootstrap starting.
I20260808 10:02:53.423009 27555 tablet_bootstrap.cc:654] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.423092 27553 tablet_bootstrap.cc:654] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.423420 27553 log.cc:824] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:53.423420 27555 log.cc:824] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:53.423970 27552 raft_consensus.cc:348] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.424154 27552 raft_consensus.cc:374] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.424202 27552 raft_consensus.cc:729] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5059dcac2eb4fd2982857ca7fa3a523, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.424202 27553 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: No bootstrap required, opened a new log
I20260808 10:02:53.424263 27553 ts_tablet_manager.cc:1397] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:02:53.424325 27552 consensus_queue.cc:260] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.424468 27555 tablet_bootstrap.cc:492] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: No bootstrap required, opened a new log
I20260808 10:02:53.424528 27555 ts_tablet_manager.cc:1397] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:02:53.424598 27552 ts_tablet_manager.cc:1428] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:02:53.424767 27523 heartbeater.cc:503] Master 127.24.162.190:44213 was elected leader, sending a full tablet report...
I20260808 10:02:53.426294 27553 raft_consensus.cc:348] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.426456 27553 raft_consensus.cc:374] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.426503 27553 raft_consensus.cc:729] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fffac3e5a74ab285b3737c3b75b572, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.426426 27555 raft_consensus.cc:348] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.426558 27555 raft_consensus.cc:374] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.426614 27555 raft_consensus.cc:729] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0a816337d64108a6ca7abac318cd5c, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.426661 27553 consensus_queue.cc:260] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.426723 27555 consensus_queue.cc:260] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.426992 27553 ts_tablet_manager.cc:1428] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:02:53.426992 27555 ts_tablet_manager.cc:1428] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:02:53.427044 27121 heartbeater.cc:503] Master 127.24.162.190:44213 was elected leader, sending a full tablet report...
I20260808 10:02:53.427033 27326 heartbeater.cc:503] Master 127.24.162.190:44213 was elected leader, sending a full tablet report...
W20260808 10:02:53.453653 27327 tablet.cc:2369] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:02:53.494201 27563 raft_consensus.cc:482] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:53.494338 27563 raft_consensus.cc:504] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.494799 27563 leader_election.cc:288] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069), d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591)
I20260808 10:02:53.499028 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.495139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:52447 (local address 127.24.162.130:42069)
0808 10:02:53.495205 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.495207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.495330 (+ 123us) server_negotiation.cc:407] Connection header received
0808 10:02:53.495359 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.495360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.495369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.495435 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.495591 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.496342 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.497936 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.498550 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.498635 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.498645 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.498881 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.498956 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:02:53.499037 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.495043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:40591 (local address 127.24.162.129:49499)
0808 10:02:53.495106 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.495119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.495166 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.495383 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.495384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.495388 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.495487 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.495489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.496448 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.496449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.497808 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.497812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.498689 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.498690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.498791 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.498794 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.498845 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.498876 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":78}
I20260808 10:02:53.499074 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.495166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:49499 (local address 127.24.162.131:40591)
0808 10:02:53.495216 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.495218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.495227 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:53.495256 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.495257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.495272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.495362 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.495586 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.496345 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.497937 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.498618 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.498695 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.498703 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.498957 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.499006 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:02:53.499053 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.495043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:42069 (local address 127.24.162.129:52447)
0808 10:02:53.495261 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.495269 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.495309 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.495448 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.495448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.495450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.495516 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.495517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.496448 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.496449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.497825 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.497827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.498632 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.498633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.498763 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.498769 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.498835 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.498876 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":105,"thread_start_us":124,"threads_started":1}
I20260808 10:02:53.499490 27461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc8fea467d3489eaf571a78858d0560" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" is_pre_election: true
I20260808 10:02:53.499682 27461 raft_consensus.cc:2454] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 0.
I20260808 10:02:53.499612 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc8fea467d3489eaf571a78858d0560" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c" is_pre_election: true
I20260808 10:02:53.499816 27257 raft_consensus.cc:2454] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 0.
I20260808 10:02:53.499939 26946 leader_election.cc:302] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fffac3e5a74ab285b3737c3b75b572, d5059dcac2eb4fd2982857ca7fa3a523; no voters:
I20260808 10:02:53.500136 27563 raft_consensus.cc:2781] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:53.500217 27563 raft_consensus.cc:482] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:53.500237 27563 raft_consensus.cc:3037] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.501269 27563 raft_consensus.cc:504] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.501416 27563 leader_election.cc:288] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 election: Requested vote from peers 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069), d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591)
I20260808 10:02:53.501579 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc8fea467d3489eaf571a78858d0560" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c"
I20260808 10:02:53.501565 27461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc8fea467d3489eaf571a78858d0560" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5059dcac2eb4fd2982857ca7fa3a523"
I20260808 10:02:53.501644 27257 raft_consensus.cc:3037] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.501667 27461 raft_consensus.cc:3037] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.502318 27257 raft_consensus.cc:2454] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 1.
I20260808 10:02:53.502427 27461 raft_consensus.cc:2454] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 1.
I20260808 10:02:53.502511 26943 leader_election.cc:302] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0a816337d64108a6ca7abac318cd5c, 66fffac3e5a74ab285b3737c3b75b572; no voters:
I20260808 10:02:53.502579 27563 raft_consensus.cc:2781] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:53.502730 27563 raft_consensus.cc:686] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 LEADER]: Becoming Leader. State: Replica: 66fffac3e5a74ab285b3737c3b75b572, State: Running, Role: LEADER
I20260808 10:02:53.502818 27563 consensus_queue.cc:237] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.503713 26810 catalog_manager.cc:5495] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "66fffac3e5a74ab285b3737c3b75b572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20260808 10:02:53.512260 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.509731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.24.162.129:37347)
0808 10:02:53.509796 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.509798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.509807 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:53.509859 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.509860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.509877 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.509942 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:53.510090 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.510542 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.511393 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.511600 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.511678 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.511898 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.511994 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.512083 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.512141 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.512163 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:02:53.513928 27461 raft_consensus.cc:1261] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Refusing update from remote peer 66fffac3e5a74ab285b3737c3b75b572: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.514338 27563 consensus_queue.cc:1035] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.515615 27257 raft_consensus.cc:1261] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Refusing update from remote peer 66fffac3e5a74ab285b3737c3b75b572: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.515936 27568 consensus_queue.cc:1035] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.517239 27572 mvcc.cc:205] Tried to move back new op lower bound from 7316207097910349824 to 7316207097868378112. Current Snapshot: MvccSnapshot[applied={T|T < 7316207097910349824}]
I20260808 10:02:53.521912 26810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:53.522023 26810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:53.523010 26810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Sending DeleteTablet for 3 replicas of tablet afc8fea467d3489eaf571a78858d0560
I20260808 10:02:53.523554 27023 tablet_service.cc:1514] Processing DeleteTablet for tablet afc8fea467d3489eaf571a78858d0560 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:59240
I20260808 10:02:53.523598 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet afc8fea467d3489eaf571a78858d0560 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:43056
I20260808 10:02:53.523567 27230 tablet_service.cc:1514] Processing DeleteTablet for tablet afc8fea467d3489eaf571a78858d0560 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:41194
I20260808 10:02:53.523837 27590 tablet_replica.cc:330] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: stopping tablet replica
I20260808 10:02:53.523929 27588 tablet_replica.cc:330] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: stopping tablet replica
I20260808 10:02:53.523880 26810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:02:53.523954 27590 raft_consensus.cc:2229] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:02:53.524008 26810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:53.524075 27590 raft_consensus.cc:2258] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.524241 27588 raft_consensus.cc:2229] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:53.524327 27591 tablet_replica.cc:330] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: stopping tablet replica
I20260808 10:02:53.524430 27591 raft_consensus.cc:2229] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:53.524580 27588 raft_consensus.cc:2258] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.525156 27590 ts_tablet_manager.cc:1905] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.525406 27591 raft_consensus.cc:2258] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.525408 27023 tablet_service.cc:1467] Processing CreateTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (DEFAULT_TABLE table=test-table [id=a449575957274233a4eecafb527429f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.525540 27023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e8f7a903a74f2da2e00ccebaa26ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.525722 27230 tablet_service.cc:1467] Processing CreateTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (DEFAULT_TABLE table=test-table [id=a449575957274233a4eecafb527429f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.525815 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e8f7a903a74f2da2e00ccebaa26ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.526827 27430 tablet_service.cc:1467] Processing CreateTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (DEFAULT_TABLE table=test-table [id=a449575957274233a4eecafb527429f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:53.526952 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e8f7a903a74f2da2e00ccebaa26ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.527098 27553 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Bootstrap starting.
I20260808 10:02:53.527392 27555 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Bootstrap starting.
I20260808 10:02:53.527585 27553 tablet_bootstrap.cc:654] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.527614 27590 ts_tablet_manager.cc:1918] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:02:53.527679 27590 log.cc:1196] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/afc8fea467d3489eaf571a78858d0560
I20260808 10:02:53.527809 27555 tablet_bootstrap.cc:654] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.527959 27590 ts_tablet_manager.cc:1939] T afc8fea467d3489eaf571a78858d0560 P 0b0a816337d64108a6ca7abac318cd5c: Deleting consensus metadata
I20260808 10:02:53.528342 26789 catalog_manager.cc:4841] TS 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069): tablet afc8fea467d3489eaf571a78858d0560 (table test-table [id=c2af354104d54fc88ad4b0543438fcf7]) successfully deleted
I20260808 10:02:53.528469 27552 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Bootstrap starting.
I20260808 10:02:53.528566 27555 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: No bootstrap required, opened a new log
I20260808 10:02:53.528610 27555 ts_tablet_manager.cc:1397] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:53.528724 27555 raft_consensus.cc:348] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.528769 27555 raft_consensus.cc:374] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.528781 27555 raft_consensus.cc:729] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0a816337d64108a6ca7abac318cd5c, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.528816 27555 consensus_queue.cc:260] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.528894 27552 tablet_bootstrap.cc:654] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.528913 27555 ts_tablet_manager.cc:1428] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.529371 27591 ts_tablet_manager.cc:1905] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.529594 27552 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: No bootstrap required, opened a new log
I20260808 10:02:53.529649 27552 ts_tablet_manager.cc:1397] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:53.529821 27552 raft_consensus.cc:348] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.529909 27552 raft_consensus.cc:374] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.529934 27552 raft_consensus.cc:729] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5059dcac2eb4fd2982857ca7fa3a523, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.529986 27552 consensus_queue.cc:260] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.530074 27552 ts_tablet_manager.cc:1428] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.531021 27591 ts_tablet_manager.cc:1918] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:02:53.531086 27588 ts_tablet_manager.cc:1905] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.531116 27591 log.cc:1196] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/afc8fea467d3489eaf571a78858d0560
I20260808 10:02:53.531358 27591 ts_tablet_manager.cc:1939] T afc8fea467d3489eaf571a78858d0560 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting consensus metadata
I20260808 10:02:53.531731 26791 catalog_manager.cc:4841] TS d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591): tablet afc8fea467d3489eaf571a78858d0560 (table test-table [id=c2af354104d54fc88ad4b0543438fcf7]) successfully deleted
I20260808 10:02:53.532653 27588 ts_tablet_manager.cc:1918] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:02:53.532729 27588 log.cc:1196] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/afc8fea467d3489eaf571a78858d0560
I20260808 10:02:53.532994 27588 ts_tablet_manager.cc:1939] T afc8fea467d3489eaf571a78858d0560 P 66fffac3e5a74ab285b3737c3b75b572: Deleting consensus metadata
I20260808 10:02:53.533398 26790 catalog_manager.cc:4841] TS 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129:37347): tablet afc8fea467d3489eaf571a78858d0560 (table test-table [id=c2af354104d54fc88ad4b0543438fcf7]) successfully deleted
I20260808 10:02:53.533644 27553 tablet_bootstrap.cc:492] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: No bootstrap required, opened a new log
I20260808 10:02:53.533708 27553 ts_tablet_manager.cc:1397] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260808 10:02:53.533861 27553 raft_consensus.cc:348] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.533947 27553 raft_consensus.cc:374] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.533972 27553 raft_consensus.cc:729] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fffac3e5a74ab285b3737c3b75b572, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.534018 27553 consensus_queue.cc:260] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.534116 27553 ts_tablet_manager.cc:1428] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:02:53.541400 27563 raft_consensus.cc:482] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:53.541476 27563 raft_consensus.cc:504] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.541621 27563 leader_election.cc:288] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069), d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591)
I20260808 10:02:53.541781 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e8f7a903a74f2da2e00ccebaa26ce3" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c" is_pre_election: true
I20260808 10:02:53.541798 27461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e8f7a903a74f2da2e00ccebaa26ce3" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" is_pre_election: true
I20260808 10:02:53.541868 27257 raft_consensus.cc:2454] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 0.
I20260808 10:02:53.541884 27461 raft_consensus.cc:2454] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 0.
I20260808 10:02:53.541996 26943 leader_election.cc:302] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0a816337d64108a6ca7abac318cd5c, 66fffac3e5a74ab285b3737c3b75b572; no voters:
I20260808 10:02:53.542092 27563 raft_consensus.cc:2781] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:53.542124 27563 raft_consensus.cc:482] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:53.542179 27563 raft_consensus.cc:3037] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.542979 27563 raft_consensus.cc:504] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.543123 27563 leader_election.cc:288] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 election: Requested vote from peers 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069), d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591)
I20260808 10:02:53.543256 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e8f7a903a74f2da2e00ccebaa26ce3" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c"
I20260808 10:02:53.543335 27257 raft_consensus.cc:3037] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.543329 27461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e8f7a903a74f2da2e00ccebaa26ce3" candidate_uuid: "66fffac3e5a74ab285b3737c3b75b572" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5059dcac2eb4fd2982857ca7fa3a523"
I20260808 10:02:53.543391 27461 raft_consensus.cc:3037] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.543890 27461 raft_consensus.cc:2454] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 1.
I20260808 10:02:53.544047 26946 leader_election.cc:302] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fffac3e5a74ab285b3737c3b75b572, d5059dcac2eb4fd2982857ca7fa3a523; no voters:
I20260808 10:02:53.544057 27257 raft_consensus.cc:2454] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66fffac3e5a74ab285b3737c3b75b572 in term 1.
I20260808 10:02:53.544236 27563 raft_consensus.cc:2781] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:53.544318 27563 raft_consensus.cc:686] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 LEADER]: Becoming Leader. State: Replica: 66fffac3e5a74ab285b3737c3b75b572, State: Running, Role: LEADER
I20260808 10:02:53.544369 27563 consensus_queue.cc:237] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.544854 26811 catalog_manager.cc:5495] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "66fffac3e5a74ab285b3737c3b75b572" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
W20260808 10:02:53.554088 27122 tablet.cc:2369] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183372, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:53.572924 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.564592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43676 (local address 127.24.162.190:44213)
0808 10:02:53.564660 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.564663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.564855 (+ 192us) server_negotiation.cc:407] Connection header received
0808 10:02:53.564898 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.564899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.564911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.564962 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.565556 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.566348 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.567436 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.567642 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.567705 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.569957 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.569959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.569959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.569966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.571569 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.571997 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.571998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.571998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.572060 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.572248 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.572249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.572249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.572383 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.572457 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.572613 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.572751 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":37}
I20260808 10:02:53.572983 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.564547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44213 (local address 127.0.0.1:43676)
0808 10:02:53.564717 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.564731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.564805 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.564977 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.564978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.565403 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:53.565462 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.565465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.566451 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.566452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.567354 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.567356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.567727 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.567728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.567789 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.568339 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:53.568344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:53.569871 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:53.571640 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.571647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.571650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.571928 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.572129 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:53.572130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:53.572130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:53.572188 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:53.572447 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:53.572454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:53.572495 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.572586 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.572804 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":83,"thread_start_us":96,"threads_started":1}
I20260808 10:02:53.575186 26811 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:43676:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:02:53.575477 26811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:53.576918 27023 tablet_service.cc:1467] Processing CreateTablet for tablet 52305cf639a7468cace223fb3bb3a04d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a361ac7351724c5498e11905a11bd85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:53.576972 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 52305cf639a7468cace223fb3bb3a04d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a361ac7351724c5498e11905a11bd85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:53.576994 27230 tablet_service.cc:1467] Processing CreateTablet for tablet 52305cf639a7468cace223fb3bb3a04d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a361ac7351724c5498e11905a11bd85e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:53.577127 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52305cf639a7468cace223fb3bb3a04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.577128 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52305cf639a7468cace223fb3bb3a04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.577129 27023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52305cf639a7468cace223fb3bb3a04d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:53.578668 27555 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: Bootstrap starting.
I20260808 10:02:53.579196 27555 tablet_bootstrap.cc:654] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.579843 27555 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: No bootstrap required, opened a new log
I20260808 10:02:53.579885 27555 ts_tablet_manager.cc:1397] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:53.580021 27555 raft_consensus.cc:348] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.580078 27555 raft_consensus.cc:374] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.580091 27555 raft_consensus.cc:729] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0a816337d64108a6ca7abac318cd5c, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.580122 27555 consensus_queue.cc:260] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.580184 27555 ts_tablet_manager.cc:1428] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.580263 27561 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.580374 27561 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.580602 27552 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: Bootstrap starting.
I20260808 10:02:53.580708 27553 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: Bootstrap starting.
I20260808 10:02:53.581120 27553 tablet_bootstrap.cc:654] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.581120 27552 tablet_bootstrap.cc:654] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:53.581971 27553 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: No bootstrap required, opened a new log
I20260808 10:02:53.582034 27553 ts_tablet_manager.cc:1397] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:02:53.582125 27552 tablet_bootstrap.cc:492] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: No bootstrap required, opened a new log
I20260808 10:02:53.582165 27552 ts_tablet_manager.cc:1397] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:02:53.582201 27553 raft_consensus.cc:348] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.582268 27553 raft_consensus.cc:374] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.582288 27553 raft_consensus.cc:729] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fffac3e5a74ab285b3737c3b75b572, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.582298 27552 raft_consensus.cc:348] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.582360 27552 raft_consensus.cc:374] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:53.582348 27553 consensus_queue.cc:260] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.582382 27552 raft_consensus.cc:729] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5059dcac2eb4fd2982857ca7fa3a523, State: Initialized, Role: FOLLOWER
I20260808 10:02:53.582425 27552 consensus_queue.cc:260] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.582448 27553 ts_tablet_manager.cc:1428] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:53.582516 27552 ts_tablet_manager.cc:1428] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:53.582470 27563 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.582557 27563 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.582633 27558 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.582706 27558 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.607551 27558 raft_consensus.cc:482] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:53.607636 27558 raft_consensus.cc:504] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.608014 27558 leader_election.cc:288] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129:37347), 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069)
I20260808 10:02:53.611639 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.608148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37347 (local address 127.24.162.131:52555)
0808 10:02:53.608288 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.608302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.608358 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.608538 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.608539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.608541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.608620 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.608623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.609364 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.609365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.610559 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.610562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.611369 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.611370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.611461 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.611465 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.611528 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.611565 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 10:02:53.611660 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.608325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:52555 (local address 127.24.162.129:37347)
0808 10:02:53.608372 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.608374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.608384 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:02:53.608413 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.608413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.608424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.608481 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.608705 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.609265 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.610685 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.611274 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.611353 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.611364 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.611551 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.611591 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:02:53.612057 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52305cf639a7468cace223fb3bb3a04d" candidate_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66fffac3e5a74ab285b3737c3b75b572" is_pre_election: true
I20260808 10:02:53.612200 27053 raft_consensus.cc:2454] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5059dcac2eb4fd2982857ca7fa3a523 in term 0.
I20260808 10:02:53.612406 27359 leader_election.cc:302] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fffac3e5a74ab285b3737c3b75b572, d5059dcac2eb4fd2982857ca7fa3a523; no voters:
I20260808 10:02:53.612547 27558 raft_consensus.cc:2781] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:53.612581 27558 raft_consensus.cc:482] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:53.612601 27558 raft_consensus.cc:3037] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.612818 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.608402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:40381 (local address 127.24.162.130:42069)
0808 10:02:53.608448 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.608450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.608644 (+ 194us) server_negotiation.cc:407] Connection header received
0808 10:02:53.608690 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.608691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.608701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.608756 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:53.608924 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.609641 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.611852 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.612421 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.612505 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.612513 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.612709 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.612752 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:02:53.612838 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.608219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:42069 (local address 127.24.162.131:40381)
0808 10:02:53.608559 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.608569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.608628 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.608773 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.608773 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.608775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:53.608844 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.608846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.609753 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.609754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.611718 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.611721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.612530 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.612532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.612619 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.612623 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.612689 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.612735 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":122,"threads_started":1}
I20260808 10:02:53.613137 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52305cf639a7468cace223fb3bb3a04d" candidate_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c" is_pre_election: true
I20260808 10:02:53.613241 27257 raft_consensus.cc:2454] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5059dcac2eb4fd2982857ca7fa3a523 in term 0.
I20260808 10:02:53.613543 27558 raft_consensus.cc:504] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.613694 27558 leader_election.cc:288] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [CANDIDATE]: Term 1 election: Requested vote from peers 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129:37347), 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069)
I20260808 10:02:53.613816 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52305cf639a7468cace223fb3bb3a04d" candidate_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66fffac3e5a74ab285b3737c3b75b572"
I20260808 10:02:53.613900 27053 raft_consensus.cc:3037] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.613871 27257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52305cf639a7468cace223fb3bb3a04d" candidate_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0a816337d64108a6ca7abac318cd5c"
I20260808 10:02:53.613953 27257 raft_consensus.cc:3037] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:53.614514 27257 raft_consensus.cc:2454] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5059dcac2eb4fd2982857ca7fa3a523 in term 1.
I20260808 10:02:53.614722 27357 leader_election.cc:302] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0a816337d64108a6ca7abac318cd5c, d5059dcac2eb4fd2982857ca7fa3a523; no voters:
I20260808 10:02:53.614766 27053 raft_consensus.cc:2454] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5059dcac2eb4fd2982857ca7fa3a523 in term 1.
I20260808 10:02:53.614826 27558 raft_consensus.cc:2781] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:53.614982 27558 raft_consensus.cc:686] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 LEADER]: Becoming Leader. State: Replica: d5059dcac2eb4fd2982857ca7fa3a523, State: Running, Role: LEADER
I20260808 10:02:53.615057 27558 consensus_queue.cc:237] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } }
I20260808 10:02:53.615442 27611 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: TxnStatusTablet state changed. Reason: New leader d5059dcac2eb4fd2982857ca7fa3a523. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.615588 27611 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:53.615887 27617 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:02:53.615844 26811 catalog_manager.cc:5495] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } health_report { overall_health: HEALTHY } } }
I20260808 10:02:53.615983 27617 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260808 10:02:53.619908 27524 tablet.cc:2369] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:02:53.631702 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.628449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.24.162.131:40591)
0808 10:02:53.628490 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.628492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.628559 (+ 67us) server_negotiation.cc:407] Connection header received
0808 10:02:53.628655 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.628656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.628675 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.628748 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:53.628869 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.629619 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.630743 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.631061 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.631133 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.631352 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.631469 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.631548 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.631611 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.631630 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:02:53.631735 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.628402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:40591 (local address 127.0.0.1:43060)
0808 10:02:53.628497 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.628526 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.628576 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.628750 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.628751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.628752 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:53.628802 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.628805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.629711 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.629713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.630648 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.630651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.631145 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.631146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.631225 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.631243 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.631548 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.631558 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.631608 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.631655 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 10:02:53.632989 27257 raft_consensus.cc:1261] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Refusing update from remote peer d5059dcac2eb4fd2982857ca7fa3a523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.633100 27053 raft_consensus.cc:1261] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Refusing update from remote peer d5059dcac2eb4fd2982857ca7fa3a523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.633237 27558 consensus_queue.cc:1035] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.633389 27558 consensus_queue.cc:1035] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.633879 27561 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: TxnStatusTablet state changed. Reason: New leader d5059dcac2eb4fd2982857ca7fa3a523. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.633960 27561 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.634009 27563 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: TxnStatusTablet state changed. Reason: New leader d5059dcac2eb4fd2982857ca7fa3a523. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.634092 27563 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.635252 27611 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.635347 27611 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:53.635517 27571 mvcc.cc:205] Tried to move back new op lower bound from 7316207098398695424 to 7316207098327732224. Current Snapshot: MvccSnapshot[applied={T|T < 7316207098398695424}]
I20260808 10:02:53.635605 27563 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.635667 27563 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 66fffac3e5a74ab285b3737c3b75b572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.637221 27558 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.637298 27558 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P d5059dcac2eb4fd2982857ca7fa3a523: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:53.637645 27577 mvcc.cc:205] Tried to move back new op lower bound from 7316207098398695424 to 7316207098327732224. Current Snapshot: MvccSnapshot[applied={T|T < 7316207098398695424}]
I20260808 10:02:53.637741 27561 tablet_replica.cc:439] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fffac3e5a74ab285b3737c3b75b572" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37347 } } peers { permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 } } peers { permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 } } }
I20260808 10:02:53.637800 27561 tablet_replica.cc:442] T 52305cf639a7468cace223fb3bb3a04d P 0b0a816337d64108a6ca7abac318cd5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:53.642535 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.639456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:33259 (local address 127.24.162.131:40591)
0808 10:02:53.639490 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.639492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.639576 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:02:53.639645 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.639646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.639656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.639713 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:53.639900 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.640618 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.641697 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.641994 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.642066 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.642283 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.642323 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.642379 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.642452 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.642469 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:02:53.642590 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.639254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:40591 (local address 127.24.162.129:33259)
0808 10:02:53.639485 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.639499 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.639593 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.639748 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.639749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.639751 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:53.639821 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.639823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.640725 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.640726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.641605 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.641608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.642100 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.642101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.642167 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.642190 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.642397 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.642399 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.642451 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.642497 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260808 10:02:53.645243 27461 raft_consensus.cc:1261] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Refusing update from remote peer 66fffac3e5a74ab285b3737c3b75b572: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.645279 27257 raft_consensus.cc:1261] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Refusing update from remote peer 66fffac3e5a74ab285b3737c3b75b572: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:53.645457 27568 consensus_queue.cc:1035] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0a816337d64108a6ca7abac318cd5c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.645540 27563 consensus_queue.cc:1035] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5059dcac2eb4fd2982857ca7fa3a523" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:53.657016 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.653643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37347 (local address 127.24.162.131:46153)
0808 10:02:53.653709 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:53.653722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:53.653760 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:53.654108 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:53.654110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:53.654111 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:53.654165 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.654166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.655062 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.655063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.655973 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:53.655976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.656466 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.656467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:53.656615 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.656631 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.656851 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.656853 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:02:53.656892 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 10:02:53.656936 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:02:53.657487 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.653911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:46153 (local address 127.24.162.129:37347)
0808 10:02:53.653978 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.653980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.653987 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:02:53.654008 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.654009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.654018 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.654067 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:53.654240 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.654979 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.656071 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.656381 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.656453 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.656745 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:53.656783 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:53.656828 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.657392 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.657419 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:02:53.670904 26811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:53.671154 26811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:53.671886 26811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6363d74eac334595a749c6815f798cce: Sending DeleteTablet for 3 replicas of tablet b8e8f7a903a74f2da2e00ccebaa26ce3
I20260808 10:02:53.672135 27023 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:59240
I20260808 10:02:53.672199 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:43056
I20260808 10:02:53.672209 27230 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e8f7a903a74f2da2e00ccebaa26ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:41194
I20260808 10:02:53.672271 27588 tablet_replica.cc:330] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: stopping tablet replica
I20260808 10:02:53.672295 27591 tablet_replica.cc:330] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: stopping tablet replica
I20260808 10:02:53.672322 27588 raft_consensus.cc:2229] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:53.672354 27591 raft_consensus.cc:2229] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:53.672379 27588 raft_consensus.cc:2258] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.672397 27591 raft_consensus.cc:2258] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.672449 27590 tablet_replica.cc:330] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: stopping tablet replica
I20260808 10:02:53.672505 27590 raft_consensus.cc:2229] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:53.672536 27590 raft_consensus.cc:2258] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:53.672708 27591 ts_tablet_manager.cc:1905] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.672853 27590 ts_tablet_manager.cc:1905] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.673126 27588 ts_tablet_manager.cc:1905] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:53.674705 27590 ts_tablet_manager.cc:1918] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:53.674774 27590 log.cc:1196] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/b8e8f7a903a74f2da2e00ccebaa26ce3
I20260808 10:02:53.674777 27588 ts_tablet_manager.cc:1918] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:53.674839 27588 log.cc:1196] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/b8e8f7a903a74f2da2e00ccebaa26ce3
I20260808 10:02:53.675043 27590 ts_tablet_manager.cc:1939] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 0b0a816337d64108a6ca7abac318cd5c: Deleting consensus metadata
I20260808 10:02:53.675072 27588 ts_tablet_manager.cc:1939] T b8e8f7a903a74f2da2e00ccebaa26ce3 P 66fffac3e5a74ab285b3737c3b75b572: Deleting consensus metadata
I20260808 10:02:53.675370 26790 catalog_manager.cc:4841] TS 66fffac3e5a74ab285b3737c3b75b572 (127.24.162.129:37347): tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (table test-table [id=a449575957274233a4eecafb527429f3]) successfully deleted
I20260808 10:02:53.675406 26789 catalog_manager.cc:4841] TS 0b0a816337d64108a6ca7abac318cd5c (127.24.162.130:42069): tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (table test-table [id=a449575957274233a4eecafb527429f3]) successfully deleted
I20260808 10:02:53.675428 27591 ts_tablet_manager.cc:1918] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:53.675477 27591 log.cc:1196] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/b8e8f7a903a74f2da2e00ccebaa26ce3
I20260808 10:02:53.675679 27591 ts_tablet_manager.cc:1939] T b8e8f7a903a74f2da2e00ccebaa26ce3 P d5059dcac2eb4fd2982857ca7fa3a523: Deleting consensus metadata
I20260808 10:02:53.676002 26791 catalog_manager.cc:4841] TS d5059dcac2eb4fd2982857ca7fa3a523 (127.24.162.131:40591): tablet b8e8f7a903a74f2da2e00ccebaa26ce3 (table test-table [id=a449575957274233a4eecafb527429f3]) successfully deleted
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[26749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:02:53.687705 27548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.674655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.24.162.129:37347)
0808 10:02:53.674699 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.674701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.676032 (+ 1331us) server_negotiation.cc:407] Connection header received
0808 10:02:53.676137 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.676138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.676152 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.676217 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.676796 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.677578 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.678282 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.678586 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.678659 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.684542 (+ 5883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.684552 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.684553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.684561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.686524 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.686874 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.686878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.686879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.686965 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.687163 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.687164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.687165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.687344 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.687485 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.687523 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.687606 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 10:02:53.688015 27112 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59260
I20260808 10:02:53.695297 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:53.689073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.24.162.190:44213)
0808 10:02:53.689127 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:53.689129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:53.689139 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:02:53.689198 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:53.689198 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:53.689210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:53.689270 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:53.689772 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.690260 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.691042 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:53.691343 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:53.691418 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:53.692593 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:53.692594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:53.692595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:53.692602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:53.694081 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.694407 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.694408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.694408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.694471 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:53.694656 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:53.694656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:53.694657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:53.694792 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:53.694896 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:53.695129 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 10:02:53.695210 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 10:02:53.695611 26811 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43686
I20260808 10:02:53.696303 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 26920
I20260808 10:02:53.723737 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 27132
I20260808 10:02:53.736634 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 27337
I20260808 10:02:53.747593 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 26766
2026-08-08T10:02:53Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2423 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[27647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[27647](info): set up 2 sockets
Aug 08 10:02:53 dist-test-slave-nfb5 krb5kdc[27647](info): commencing operation
krb5kdc: starting...
W20260808 10:02:54.978884 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:02:55.102108 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33629
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:55.185552 27667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:55.185768 27667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:55.185792 27667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:55.187091 27667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:02:55.187134 27667 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:02:55.187147 27667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:55.187181 27667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:55.187214 27667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:02:55.187232 27667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:02:55.188714 27667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33629
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:55.188978 27667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:55.189229 27667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:55.194193 27675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.194283 27667 server_base.cc:1034] running on GCE node
W20260808 10:02:55.194417 27680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:55.194463 27674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.194727 27667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:55.195083 27667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:55.196249 27667 hybrid_clock.cc:648] HybridClock initialized: now 1786183375196236 us; error 32 us; skew 500 ppm
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:55.198729 27667 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:02:55.199193 27667 webserver.cc:460] Webserver started at http://127.24.162.190:34169/ using document root <none> and password file <none>
I20260808 10:02:55.199450 27667 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:55.199520 27667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:55.199668 27667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:55.200737 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "f90c5ef309f143b6ad96b566065a9c18"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "81a27ae6bbc8c37c5cea3e2f8190d27c"
server_key_iv: "d9e1e6094d4720b7288e22b01398cfa9"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.201109 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "f90c5ef309f143b6ad96b566065a9c18"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "81a27ae6bbc8c37c5cea3e2f8190d27c"
server_key_iv: "d9e1e6094d4720b7288e22b01398cfa9"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.202482 27667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:02:55.203284 27692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.203461 27667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.203574 27667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "f90c5ef309f143b6ad96b566065a9c18"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "81a27ae6bbc8c37c5cea3e2f8190d27c"
server_key_iv: "d9e1e6094d4720b7288e22b01398cfa9"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.203650 27667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:55.212759 27667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:55.214588 27667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:55.214753 27667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:55.219693 27667 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33629
I20260808 10:02:55.219754 27791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33629 every 8 connection(s)
I20260808 10:02:55.220067 27667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:02:55.220703 27793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.223067 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Bootstrap starting.
I20260808 10:02:55.223762 27793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.224059 27793 log.cc:824] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:55.224754 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: No bootstrap required, opened a new log
I20260808 10:02:55.226850 27793 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } }
I20260808 10:02:55.227089 27793 raft_consensus.cc:374] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.227149 27793 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90c5ef309f143b6ad96b566065a9c18, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.227232 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 27667
I20260808 10:02:55.227315 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:02:55.227324 27793 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } }
I20260808 10:02:55.227419 27793 raft_consensus.cc:388] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:02:55.227460 27793 raft_consensus.cc:482] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:02:55.227499 25226 external_mini_cluster.cc:1428] Setting key ab8850cc91e2e95676c01405abbaf856
I20260808 10:02:55.227511 27793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.228189 27793 raft_consensus.cc:504] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } }
I20260808 10:02:55.228327 27793 leader_election.cc:302] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f90c5ef309f143b6ad96b566065a9c18; no voters:
I20260808 10:02:55.228508 27793 leader_election.cc:288] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:02:55.228542 27799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:55.228732 27799 raft_consensus.cc:686] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [term 1 LEADER]: Becoming Leader. State: Replica: f90c5ef309f143b6ad96b566065a9c18, State: Running, Role: LEADER
I20260808 10:02:55.228793 27793 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:02:55.228924 27799 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } }
I20260808 10:02:55.229393 27799 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90c5ef309f143b6ad96b566065a9c18. Latest consensus state: current_term: 1 leader_uuid: "f90c5ef309f143b6ad96b566065a9c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } } }
I20260808 10:02:55.229544 27799 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:55.229775 27801 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f90c5ef309f143b6ad96b566065a9c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90c5ef309f143b6ad96b566065a9c18" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33629 } } }
I20260808 10:02:55.229808 27812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:02:55.229893 27801 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18 [sys.catalog]: This master's current role is: LEADER
I20260808 10:02:55.230736 27812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:02:55.232547 27812 catalog_manager.cc:1347] Generated new cluster ID: 3174eb8d90af4897a7205661b4f1b199
I20260808 10:02:55.232604 27812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.240934 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.228007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.24.162.190:33629)
0808 10:02:55.228221 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.228231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.228264 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:02:55.228437 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.228445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.228502 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.229484 (+ 982us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.229587 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.231396 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.232388 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.233020 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.233134 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.236177 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.236181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.236182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.236195 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.238987 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.239400 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.239404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.239404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.239526 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.239739 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.239740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.239740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.240071 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.240234 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.240509 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.240677 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260808 10:02:55.240974 27812 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:02:55.241792 27812 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:02:55.246569 27812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Generated new TSK 0
I20260808 10:02:55.246750 27812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:02:55.279503 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33629
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:55.362136 27831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:55.362347 27831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:55.362373 27831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:55.363575 27831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:55.363620 27831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:55.363642 27831 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:02:55.363696 27831 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:55.363722 27831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:02:55.365196 27831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:55.365448 27831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:55.365712 27831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:55.370993 27837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:55.370993 27839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:55.371002 27842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.371094 27831 server_base.cc:1034] running on GCE node
I20260808 10:02:55.371428 27831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:55.371639 27831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:55.372781 27831 hybrid_clock.cc:648] HybridClock initialized: now 1786183375372766 us; error 30 us; skew 500 ppm
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:55.375464 27831 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:02:55.375938 27831 webserver.cc:460] Webserver started at http://127.24.162.129:37099/ using document root <none> and password file <none>
I20260808 10:02:55.376163 27831 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:55.376207 27831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:55.376380 27831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:55.377451 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "9954c8e3de8949a7ad55fe0111f45911"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "870686a61dfed9317b5f88c4b412ab66"
server_key_iv: "cc6393421bf37ba40e406508f0b140c4"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.377866 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "9954c8e3de8949a7ad55fe0111f45911"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "870686a61dfed9317b5f88c4b412ab66"
server_key_iv: "cc6393421bf37ba40e406508f0b140c4"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.379769 27831 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 10:02:55.380527 27851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.380718 27831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.380839 27831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "9954c8e3de8949a7ad55fe0111f45911"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "870686a61dfed9317b5f88c4b412ab66"
server_key_iv: "cc6393421bf37ba40e406508f0b140c4"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.380926 27831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:55.391961 27831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:55.394446 27831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:55.394613 27831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:55.395264 27831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:55.395326 27831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.395360 27831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:55.395390 27831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.402864 27831 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:45617
I20260808 10:02:55.402976 28029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:45617 every 8 connection(s)
I20260808 10:02:55.403313 27831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:02:55.404770 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 27831
I20260808 10:02:55.404878 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:02:55.404935 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.395658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.129:54835)
0808 10:02:55.395845 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.395886 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.396403 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.396663 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.396666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.396977 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.397156 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.397160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.397880 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.397881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.398666 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.398669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.399199 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.399201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.399333 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.399894 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.399902 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.401717 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.403690 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.403697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.403699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.403954 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.404148 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.404148 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.404149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.404245 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.404444 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.404446 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.404549 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.404624 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.404737 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260808 10:02:55.404947 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.395662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:54835 (local address 127.24.162.190:33629)
0808 10:02:55.395738 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.395740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.395985 (+ 245us) server_negotiation.cc:407] Connection header received
0808 10:02:55.396508 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.396509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.396523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.396591 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.397265 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.397772 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.398800 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.399109 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.399191 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.401824 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.401826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.401826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.401846 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.403585 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.404023 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.404024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.404025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.404080 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.404300 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.404300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.404301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.404403 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.404454 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.404674 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.404828 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:02:55.405077 25226 external_mini_cluster.cc:1428] Setting key ad2cac8c37d4f31b5175a2ee9e38814c
I20260808 10:02:55.405702 28033 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33629
I20260808 10:02:55.405791 28033 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:55.406038 28033 heartbeater.cc:511] Master 127.24.162.190:33629 requested a full tablet report, sending...
I20260808 10:02:55.406615 27716 ts_manager.cc:194] Registered new tserver with Master: 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617)
I20260808 10:02:55.407563 27716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:54835
I20260808 10:02:55.413034 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.406558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.129:42189)
0808 10:02:55.406721 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.406732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.406800 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.406954 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.406955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.407135 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.407185 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.407186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.408115 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.408116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.408806 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.408809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.409299 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.409300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.409357 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.409793 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.409797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.410127 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.412021 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.412022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.412022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.412238 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.412453 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.412454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.412454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.412508 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.412796 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.412797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.412842 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.412895 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.412970 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":27,"thread_start_us":74,"threads_started":1}
I20260808 10:02:55.413146 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.406609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:42189 (local address 127.24.162.190:33629)
0808 10:02:55.406659 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.406661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.406786 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:02:55.406859 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.406859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.406869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.406948 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.407287 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.408011 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.408905 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.409215 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.409295 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.410233 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.410234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.410235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.410243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.411939 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.412318 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.412319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.412320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.412385 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.412575 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.412575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.412576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.412716 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.412804 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.412942 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.413028 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:02:55.433365 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33629
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:55.515856 28045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:55.516104 28045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:55.516156 28045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:55.517307 28045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:55.517380 28045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:55.517417 28045 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:02:55.517447 28045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:55.517480 28045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:02:55.518987 28045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:55.519276 28045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:55.519515 28045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:55.524645 28052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:55.524657 28056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.524688 28045 server_base.cc:1034] running on GCE node
W20260808 10:02:55.524650 28054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.525050 28045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:55.525319 28045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:55.526507 28045 hybrid_clock.cc:648] HybridClock initialized: now 1786183375526502 us; error 42 us; skew 500 ppm
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:55.529198 28045 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:02:55.529717 28045 webserver.cc:460] Webserver started at http://127.24.162.130:34781/ using document root <none> and password file <none>
I20260808 10:02:55.529978 28045 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:55.530048 28045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:55.530280 28045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:55.531704 28045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "a200102e32e040b2a43cdae4748cd19c"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "ffffe0c53279c3402266efb197b32843"
server_key_iv: "e8934146eb031685b22f7c9aa4296ece"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.532008 28045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "a200102e32e040b2a43cdae4748cd19c"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "ffffe0c53279c3402266efb197b32843"
server_key_iv: "e8934146eb031685b22f7c9aa4296ece"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.533294 28045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:02:55.534019 28067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.534214 28045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.534315 28045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "a200102e32e040b2a43cdae4748cd19c"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "ffffe0c53279c3402266efb197b32843"
server_key_iv: "e8934146eb031685b22f7c9aa4296ece"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.534406 28045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:55.551121 28045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:55.553045 28045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:55.553220 28045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:55.553802 28045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:55.553864 28045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.553926 28045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:55.553974 28045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.561606 28045 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:33183
I20260808 10:02:55.561736 28244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:33183 every 8 connection(s)
I20260808 10:02:55.562031 28045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:02:55.564273 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.554199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:44579 (local address 127.24.162.190:33629)
0808 10:02:55.554269 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.554271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.554546 (+ 275us) server_negotiation.cc:407] Connection header received
0808 10:02:55.555328 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.555329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.555340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.555381 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.556303 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.556858 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.558529 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.558742 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.558845 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.561088 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.561090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.561090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.561097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.562777 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.563272 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.563274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.563274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.563348 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.563619 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.563621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.563621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.563741 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.563820 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.564097 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.564186 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:02:55.564435 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.554185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.130:44579)
0808 10:02:55.554396 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.554442 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.555217 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.555464 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.555468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.555886 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.556190 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.556198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.556941 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.556942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.558408 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.558412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.558868 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.558869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.559052 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.559570 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.559581 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.560980 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.562877 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.562884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.562886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.563191 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.563417 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.563418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.563418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.563544 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.563808 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.563822 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.563958 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.564060 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.564213 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260808 10:02:55.564915 28245 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33629
I20260808 10:02:55.565003 28245 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:55.565200 28245 heartbeater.cc:511] Master 127.24.162.190:33629 requested a full tablet report, sending...
I20260808 10:02:55.565562 27716 ts_manager.cc:194] Registered new tserver with Master: a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183)
I20260808 10:02:55.566071 27716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:44579
I20260808 10:02:55.568383 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 28045
I20260808 10:02:55.568504 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:02:55.569033 25226 external_mini_cluster.cc:1428] Setting key d5d5caef1853e96a084cc59bbd990269
I20260808 10:02:55.574254 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.566081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:38239 (local address 127.24.162.190:33629)
0808 10:02:55.566126 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.566128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.566197 (+ 69us) server_negotiation.cc:407] Connection header received
0808 10:02:55.566255 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.566256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.566273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.566324 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.566664 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.567244 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.567916 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.568090 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.568160 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.569249 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.569251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.569251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.569258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.572984 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.573327 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.573329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.573329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.573395 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.573649 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.573651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.573651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.573865 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.573937 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.574126 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.574195 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:02:55.574361 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.566005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.130:38239)
0808 10:02:55.566133 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.566144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.566207 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.566338 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.566339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.566532 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.566581 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.566583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.567361 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.567362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.567839 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.567840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.568202 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.568203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.568286 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.568801 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.568806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.569175 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.573074 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.573075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.573075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.573245 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.573463 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.573463 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.573464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.573544 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.573967 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.573969 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.574018 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.574088 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.574209 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:02:55.597565 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33629
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:02:55.679692 28254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:02:55.679925 28254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:02:55.679965 28254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:02:55.681118 28254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:02:55.681190 28254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:02:55.681226 28254 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:02:55.681257 28254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:02:55.681298 28254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:02:55.682747 28254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:02:55.683030 28254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:02:55.683274 28254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:02:55.688299 28272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.688371 28254 server_base.cc:1034] running on GCE node
W20260808 10:02:55.688272 28268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:02:55.688283 28267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:02:55.688846 28254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:02:55.689088 28254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:02:55.690250 28254 hybrid_clock.cc:648] HybridClock initialized: now 1786183375690253 us; error 43 us; skew 500 ppm
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:02:55.692723 28254 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:02:55.693163 28254 webserver.cc:460] Webserver started at http://127.24.162.131:42885/ using document root <none> and password file <none>
I20260808 10:02:55.693342 28254 fs_manager.cc:359] Metadata directory not provided
I20260808 10:02:55.693382 28254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:02:55.693471 28254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:02:55.694453 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "c3697f8323312799371ac3278dc1eaed"
server_key_iv: "523196f11953c98d2b6398b435b14fca"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.694747 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "c3697f8323312799371ac3278dc1eaed"
server_key_iv: "523196f11953c98d2b6398b435b14fca"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.696070 28254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:02:55.696723 28281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.696907 28254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.697000 28254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6"
format_stamp: "Formatted at 2026-08-08 10:02:55 on dist-test-slave-nfb5"
server_key: "c3697f8323312799371ac3278dc1eaed"
server_key_iv: "523196f11953c98d2b6398b435b14fca"
server_key_version: "encryptionkey@0"
I20260808 10:02:55.697052 28254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:02:55.705448 28254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:02:55.707662 28254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:02:55.707779 28254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:02:55.708199 28254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:02:55.708237 28254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.708288 28254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:02:55.708315 28254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.715371 28254 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:41413
I20260808 10:02:55.715436 28437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:41413 every 8 connection(s)
I20260808 10:02:55.715750 28254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:02:55.718420 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.708692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:34787 (local address 127.24.162.190:33629)
0808 10:02:55.708735 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.708737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.708950 (+ 213us) server_negotiation.cc:407] Connection header received
0808 10:02:55.709401 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.709402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.709415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.709493 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.710319 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.711224 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.712681 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.713020 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.713111 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.715927 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.715929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.715929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.715939 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.717162 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.717539 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.717540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.717540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.717592 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.717777 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.717778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.717778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.717883 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.717935 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.718281 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.718356 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:02:55.718633 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.708680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.131:34787)
0808 10:02:55.708848 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.708880 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.709303 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.709556 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.709559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.709930 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.710207 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.710212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.711334 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.711335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.712609 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.712612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.713114 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.713115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.713262 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.713819 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.713828 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.715817 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.717239 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.717244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.717246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.717464 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.717635 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.717636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.717636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.717724 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.718007 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.718013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.718141 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.718259 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.718397 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 10:02:55.719245 28439 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33629
I20260808 10:02:55.719334 28439 heartbeater.cc:464] Registering TS with master...
I20260808 10:02:55.719547 28439 heartbeater.cc:511] Master 127.24.162.190:33629 requested a full tablet report, sending...
I20260808 10:02:55.720010 27714 ts_manager.cc:194] Registered new tserver with Master: 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413)
I20260808 10:02:55.720547 27714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:34787
I20260808 10:02:55.722767 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 28254
I20260808 10:02:55.722884 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:02:55.723133 25226 external_mini_cluster.cc:1428] Setting key e94355a9091b0db31d30e90da7ebc0c7
I20260808 10:02:55.724349 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:02:55.726280 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.720339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:43003 (local address 127.24.162.190:33629)
0808 10:02:55.720390 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.720392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.720501 (+ 109us) server_negotiation.cc:407] Connection header received
0808 10:02:55.720528 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.720529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.720539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.720608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.721087 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.721588 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.722336 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.722509 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.722561 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.723633 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.723634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.723634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.723640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.725189 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.725614 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.725614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.725615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.725665 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.725853 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.725854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.725854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.725946 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.725997 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.726162 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.726222 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:02:55.726428 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.720265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.24.162.131:43003)
0808 10:02:55.720411 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.720424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.720491 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.720633 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.720634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.720912 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.720989 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.720991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.721748 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.721749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.722257 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.722259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.722597 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.722598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.722680 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.723176 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.723188 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.723539 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.725275 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.725276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.725277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.725513 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.725719 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.725720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.725720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.725790 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.726013 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.726014 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.726063 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.726147 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.726278 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.748629 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.741798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.24.162.190:33629)
0808 10:02:55.741869 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.741871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.741879 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:02:55.741900 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.741901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.741909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.741958 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.742487 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.743073 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.743771 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.743955 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.744004 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.746170 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.746172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.746173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.746180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.747694 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.748017 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.748017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.748018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.748070 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.748237 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.748238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.748238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.748334 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.748408 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.748508 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.748565 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:02:55.750380 27714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:02:55.752038 27714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:55.758466 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:41413 (local address 127.0.0.1:51548)
0808 10:02:55.754480 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.754491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.754535 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.754981 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.754982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.754983 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:55.755023 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.755025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.756038 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.756039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.757351 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.757354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758097 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.758181 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758194 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.758311 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.758378 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260808 10:02:55.758476 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.24.162.131:41413)
0808 10:02:55.754709 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.754713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.754723 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:02:55.754762 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.754764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.754781 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.754954 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:55.755086 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.755955 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.757448 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758007 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758100 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758163 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.758332 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.758378 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":81,"threads_started":1}
I20260808 10:02:55.758822 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.24.162.129:45617)
0808 10:02:55.754807 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.754810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.755238 (+ 428us) server_negotiation.cc:407] Connection header received
0808 10:02:55.755282 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.755284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.755299 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.755376 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:55.755522 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.756481 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.757586 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758294 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758361 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758410 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.758661 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.758725 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260808 10:02:55.758616 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:33183 (local address 127.0.0.1:34710)
0808 10:02:55.754366 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.754412 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.754482 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.754932 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.754935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.754938 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:55.755002 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.755004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.756292 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.756293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.757351 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.757354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758359 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.758437 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758442 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.758491 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.758538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":46,"threads_started":1}
I20260808 10:02:55.758842 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.24.162.130:33183)
0808 10:02:55.754587 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.754592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.754605 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:02:55.754761 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.754769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.754782 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.754866 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:55.755095 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.756203 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.757458 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758177 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758263 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758321 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.758704 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.758745 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260808 10:02:55.758713 28464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.754392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45617 (local address 127.0.0.1:46482)
0808 10:02:55.755123 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.755134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.755179 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.755394 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.755395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.755397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:55.755438 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.755439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.756570 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.756571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.757496 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.757499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.758442 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.758443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.758506 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.758508 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.758551 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.758582 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":80,"threads_started":1}
I20260808 10:02:55.759613 28148 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c (DEFAULT_TABLE table=test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.759611 28345 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c (DEFAULT_TABLE table=test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.760000 28345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a9a354cda4000a8becb7a6337f47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.760000 28148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a9a354cda4000a8becb7a6337f47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.762885 27925 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c (DEFAULT_TABLE table=test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.763286 27925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a9a354cda4000a8becb7a6337f47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.763461 28471 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Bootstrap starting.
I20260808 10:02:55.763829 28477 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Bootstrap starting.
I20260808 10:02:55.764389 28477 tablet_bootstrap.cc:654] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.764436 28471 tablet_bootstrap.cc:654] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.764681 28477 log.cc:824] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:55.764791 28471 log.cc:824] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:55.765386 28477 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: No bootstrap required, opened a new log
I20260808 10:02:55.765444 28477 ts_tablet_manager.cc:1397] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:02:55.766961 28482 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Bootstrap starting.
I20260808 10:02:55.767195 28477 raft_consensus.cc:348] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.767382 28477 raft_consensus.cc:374] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.767417 28477 raft_consensus.cc:729] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.767545 28471 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: No bootstrap required, opened a new log
I20260808 10:02:55.767537 28477 consensus_queue.cc:260] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.767608 28471 ts_tablet_manager.cc:1397] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:02:55.767614 28482 tablet_bootstrap.cc:654] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.767829 28477 ts_tablet_manager.cc:1428] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:02:55.767849 28245 heartbeater.cc:503] Master 127.24.162.190:33629 was elected leader, sending a full tablet report...
I20260808 10:02:55.767964 28482 log.cc:824] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Log is configured to *not* fsync() on all Append() calls
I20260808 10:02:55.769016 28482 tablet_bootstrap.cc:492] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: No bootstrap required, opened a new log
I20260808 10:02:55.769093 28482 ts_tablet_manager.cc:1397] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:02:55.769503 28471 raft_consensus.cc:348] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.769652 28471 raft_consensus.cc:374] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.769698 28471 raft_consensus.cc:729] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.769807 28471 consensus_queue.cc:260] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.770087 28471 ts_tablet_manager.cc:1428] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:02:55.770226 28439 heartbeater.cc:503] Master 127.24.162.190:33629 was elected leader, sending a full tablet report...
I20260808 10:02:55.771121 28482 raft_consensus.cc:348] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.771260 28482 raft_consensus.cc:374] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.771304 28482 raft_consensus.cc:729] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954c8e3de8949a7ad55fe0111f45911, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.771415 28482 consensus_queue.cc:260] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.771653 28482 ts_tablet_manager.cc:1428] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:02:55.771701 28033 heartbeater.cc:503] Master 127.24.162.190:33629 was elected leader, sending a full tablet report...
I20260808 10:02:55.772054 28485 raft_consensus.cc:482] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:55.772140 28485 raft_consensus.cc:504] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.772512 28485 leader_election.cc:288] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183), 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617)
I20260808 10:02:55.775424 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.772700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:52359 (local address 127.24.162.129:45617)
0808 10:02:55.772737 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.772739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.772872 (+ 133us) server_negotiation.cc:407] Connection header received
0808 10:02:55.773006 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.773007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.773014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.773040 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:55.773233 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.773786 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.774659 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.775097 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.775171 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.775178 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.775329 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.775362 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:02:55.775481 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.772660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45617 (local address 127.24.162.131:52359)
0808 10:02:55.772794 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.772803 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.772881 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.773100 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.773101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.773102 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:55.773172 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.773173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.773863 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.773864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.774580 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.774581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.775177 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.775178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.775256 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.775261 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.775323 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.775362 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260808 10:02:55.775497 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.772625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:33183 (local address 127.24.162.131:33155)
0808 10:02:55.772663 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.772676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.772717 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.772972 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.772973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.772974 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:55.773024 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.773026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.773693 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.773694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.774455 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.774457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.775170 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.775171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.775244 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.775270 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.775325 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.775369 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 10:02:55.775568 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.772799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:33155 (local address 127.24.162.130:33183)
0808 10:02:55.772865 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.772867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.772876 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:55.772898 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.772898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.772907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.772963 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:55.773093 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.773616 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.774563 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.775034 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.775122 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.775132 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.775464 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.775502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:02:55.775827 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a9a354cda4000a8becb7a6337f47c" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c" is_pre_election: true
I20260808 10:02:55.775902 27949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a9a354cda4000a8becb7a6337f47c" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911" is_pre_election: true
I20260808 10:02:55.775966 28178 raft_consensus.cc:2454] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.776006 27949 raft_consensus.cc:2454] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.776146 28285 leader_election.cc:302] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:55.776271 28485 raft_consensus.cc:2781] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:55.776335 28485 raft_consensus.cc:482] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:55.776364 28485 raft_consensus.cc:3037] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.776976 28485 raft_consensus.cc:504] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.777117 28485 leader_election.cc:288] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Requested vote from peers a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183), 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617)
I20260808 10:02:55.777247 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a9a354cda4000a8becb7a6337f47c" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c"
I20260808 10:02:55.777287 27949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a9a354cda4000a8becb7a6337f47c" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911"
I20260808 10:02:55.777338 28178 raft_consensus.cc:3037] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.777360 27949 raft_consensus.cc:3037] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.777920 27949 raft_consensus.cc:2454] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.778051 28178 raft_consensus.cc:2454] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.778093 28284 leader_election.cc:302] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, 9954c8e3de8949a7ad55fe0111f45911; no voters:
I20260808 10:02:55.778167 28485 raft_consensus.cc:2781] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:55.778301 28485 raft_consensus.cc:686] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Running, Role: LEADER
I20260808 10:02:55.778398 28485 consensus_queue.cc:237] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } }
I20260808 10:02:55.779201 27714 catalog_manager.cc:5495] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } health_report { overall_health: UNKNOWN } } }
W20260808 10:02:55.812624 28246 tablet.cc:2369] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:02:55 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:55.818809 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.811708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.24.162.190:33629)
0808 10:02:55.811792 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.811795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.811906 (+ 111us) server_negotiation.cc:407] Connection header received
0808 10:02:55.811999 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.811999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.812010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.812084 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:55.812554 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.813103 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.813847 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.814031 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.814103 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.816258 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:55.816260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:55.816260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:55.816267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:55.817574 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.817952 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.817953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.817953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.818025 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:55.818218 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:55.818219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:55.818219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:55.818327 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:55.818378 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.818549 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.818663 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":49}
I20260808 10:02:55.818815 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.811625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33629 (local address 127.0.0.1:48834)
0808 10:02:55.811831 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.811840 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.811926 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.812090 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.812091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.812412 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:02:55.812459 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.812461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.813195 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.813196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.813748 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.813750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.814100 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.814101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.814150 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.814717 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:02:55.814722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:02:55.816153 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:02:55.817663 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.817670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.817672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.817882 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.818089 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:02:55.818090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:02:55.818090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:02:55.818161 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:02:55.818402 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:02:55.818408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:02:55.818455 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.818516 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.818613 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260808 10:02:55.820930 27714 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:48834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:02:55.821107 27714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:55.822547 27925 tablet_service.cc:1467] Processing CreateTablet for tablet 95505350d1344e1d965ffa5c56166ed8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3215ef3299964000a657542190043ae3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:55.822605 28148 tablet_service.cc:1467] Processing CreateTablet for tablet 95505350d1344e1d965ffa5c56166ed8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3215ef3299964000a657542190043ae3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:55.822765 27925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95505350d1344e1d965ffa5c56166ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.822806 28148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95505350d1344e1d965ffa5c56166ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.822906 28345 tablet_service.cc:1467] Processing CreateTablet for tablet 95505350d1344e1d965ffa5c56166ed8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3215ef3299964000a657542190043ae3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:02:55.823076 28345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95505350d1344e1d965ffa5c56166ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.824230 28477 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: Bootstrap starting.
I20260808 10:02:55.824517 28482 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: Bootstrap starting.
I20260808 10:02:55.824606 28471 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Bootstrap starting.
I20260808 10:02:55.824947 28477 tablet_bootstrap.cc:654] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.825065 28471 tablet_bootstrap.cc:654] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.825357 28482 tablet_bootstrap.cc:654] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.825742 28477 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: No bootstrap required, opened a new log
I20260808 10:02:55.825794 28477 ts_tablet_manager.cc:1397] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:02:55.825971 28477 raft_consensus.cc:348] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.826050 28477 raft_consensus.cc:374] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.826076 28477 raft_consensus.cc:729] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.826136 28477 consensus_queue.cc:260] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.826249 28477 ts_tablet_manager.cc:1428] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.826336 28483 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.826431 28483 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.827217 28471 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: No bootstrap required, opened a new log
I20260808 10:02:55.827292 28471 ts_tablet_manager.cc:1397] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:02:55.827304 28482 tablet_bootstrap.cc:492] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: No bootstrap required, opened a new log
I20260808 10:02:55.827350 28482 ts_tablet_manager.cc:1397] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:02:55.827443 28471 raft_consensus.cc:348] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.827536 28471 raft_consensus.cc:374] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.827505 28482 raft_consensus.cc:348] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.827571 28471 raft_consensus.cc:729] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.827592 28482 raft_consensus.cc:374] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.827620 28482 raft_consensus.cc:729] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954c8e3de8949a7ad55fe0111f45911, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.827620 28471 consensus_queue.cc:260] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.827706 28471 ts_tablet_manager.cc:1428] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.827662 28482 consensus_queue.cc:260] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.827801 28482 ts_tablet_manager.cc:1428] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.827754 28485 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.827836 28485 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.827872 28486 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.827971 28486 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.880664 28485 raft_consensus.cc:482] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:55.880771 28485 raft_consensus.cc:504] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.880918 28485 leader_election.cc:288] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183)
I20260808 10:02:55.881141 27949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95505350d1344e1d965ffa5c56166ed8" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911" is_pre_election: true
I20260808 10:02:55.881170 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95505350d1344e1d965ffa5c56166ed8" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c" is_pre_election: true
I20260808 10:02:55.881261 28178 raft_consensus.cc:2454] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.881261 27949 raft_consensus.cc:2454] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.881402 28284 leader_election.cc:302] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, 9954c8e3de8949a7ad55fe0111f45911; no voters:
I20260808 10:02:55.881516 28485 raft_consensus.cc:2781] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:55.881578 28485 raft_consensus.cc:482] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:55.881593 28485 raft_consensus.cc:3037] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.882287 28485 raft_consensus.cc:504] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.882429 28485 leader_election.cc:288] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Requested vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183)
I20260808 10:02:55.882617 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95505350d1344e1d965ffa5c56166ed8" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c"
I20260808 10:02:55.882630 27949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95505350d1344e1d965ffa5c56166ed8" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911"
I20260808 10:02:55.882722 27949 raft_consensus.cc:3037] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.882728 28178 raft_consensus.cc:3037] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.883432 28178 raft_consensus.cc:2454] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.883440 27949 raft_consensus.cc:2454] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.883632 28284 leader_election.cc:302] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, 9954c8e3de8949a7ad55fe0111f45911; no voters:
I20260808 10:02:55.883733 28485 raft_consensus.cc:2781] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:55.883802 28485 raft_consensus.cc:686] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Running, Role: LEADER
I20260808 10:02:55.883857 28485 consensus_queue.cc:237] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.884195 28489 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: TxnStatusTablet state changed. Reason: New leader 1fe5a15d6da648b2b36e3ecc188d6cb6. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.884317 28489 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:55.884550 28503 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:02:55.884524 27716 catalog_manager.cc:5495] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } health_report { overall_health: HEALTHY } } }
I20260808 10:02:55.884682 28503 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:02:55.900350 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.897556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.24.162.131:41413)
0808 10:02:55.897619 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.897621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.897695 (+ 74us) server_negotiation.cc:407] Connection header received
0808 10:02:55.897789 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.897790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.897806 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.897871 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:55.898006 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.898458 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.899583 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.899762 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.899839 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.900038 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:55.900124 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:55.900179 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.900253 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.900269 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:02:55.900377 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.897509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:41413 (local address 127.0.0.1:51556)
0808 10:02:55.897631 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.897644 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.897714 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.897890 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.897891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.897893 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:55.897946 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.897948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.898532 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.898533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.899505 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.899508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.899851 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.899852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.899922 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.899941 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:55.900185 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:55.900193 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.900251 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.900297 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 10:02:55.902231 28178 raft_consensus.cc:1261] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.902238 27949 raft_consensus.cc:1261] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.902558 28489 consensus_queue.cc:1035] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:55.902699 28485 consensus_queue.cc:1035] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:02:55.904006 28034 tablet.cc:2369] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:02:55.904026 28486 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: TxnStatusTablet state changed. Reason: New leader 1fe5a15d6da648b2b36e3ecc188d6cb6. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.904166 28486 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.904402 28483 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: TxnStatusTablet state changed. Reason: New leader 1fe5a15d6da648b2b36e3ecc188d6cb6. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.904515 28483 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.905561 28489 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.905665 28489 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:55.905788 28486 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.905870 28486 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 9954c8e3de8949a7ad55fe0111f45911: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.906242 28507 mvcc.cc:205] Tried to move back new op lower bound from 7316207107693256704 to 7316207107620712448. Current Snapshot: MvccSnapshot[applied={T|T < 7316207107693256704}]
I20260808 10:02:55.906538 28489 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.906617 28489 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P 1fe5a15d6da648b2b36e3ecc188d6cb6: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:02:55.907399 28483 tablet_replica.cc:439] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } } }
I20260808 10:02:55.907490 28483 tablet_replica.cc:442] T 95505350d1344e1d965ffa5c56166ed8 P a200102e32e040b2a43cdae4748cd19c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:02:55.910859 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.908512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.24.162.131:41413)
0808 10:02:55.908552 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.908554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.908563 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:55.908585 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.908586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.908595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.908660 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:55.908743 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.909445 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.910168 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.910475 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.910549 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.910648 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:55.910680 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:55.910732 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.910777 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.910792 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:02:55.915904 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.913347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:41413 (local address 127.24.162.131:60915)
0808 10:02:55.913432 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:55.913441 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:55.913477 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:55.913594 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:55.913594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:55.913595 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:55.913638 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.913639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.914451 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.914451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.915072 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:55.915074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.915512 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.915513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:55.915559 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.915572 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:55.915751 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:55.915753 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:02:55.915787 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 10:02:55.915817 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:02:55.915905 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:55.913390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:60915 (local address 127.24.162.131:41413)
0808 10:02:55.913428 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:55.913430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:55.913471 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:02:55.913515 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:55.913515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:55.913523 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:55.913587 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:55.913674 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.914380 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.915148 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:55.915435 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:55.915508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:55.915655 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:55.915691 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:55.915744 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:55.915790 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 10:02:55.915808 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:02:55.918181 27949 raft_consensus.cc:1261] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.918197 28178 raft_consensus.cc:1261] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.918416 28485 consensus_queue.cc:1035] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:55.918501 28489 consensus_queue.cc:1035] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:55.919590 28506 mvcc.cc:205] Tried to move back new op lower bound from 7316207107759640576 to 7316207107189075968. Current Snapshot: MvccSnapshot[applied={T|T < 7316207107759640576}]
I20260808 10:02:55.920199 28509 mvcc.cc:205] Tried to move back new op lower bound from 7316207107759640576 to 7316207107189075968. Current Snapshot: MvccSnapshot[applied={T|T < 7316207107759640576}]
I20260808 10:02:55.932611 27714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:55.932726 27714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:55.933476 27714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Sending DeleteTablet for 3 replicas of tablet 1e8a9a354cda4000a8becb7a6337f47c
I20260808 10:02:55.933785 28345 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:55 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:51548
I20260808 10:02:55.933794 28148 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:55 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:34710
I20260808 10:02:55.933794 27925 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a9a354cda4000a8becb7a6337f47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:55 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46482
I20260808 10:02:55.934023 27714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:02:55.934163 28530 tablet_replica.cc:330] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: stopping tablet replica
W20260808 10:02:55.934176 27714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:55.934260 28530 raft_consensus.cc:2229] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:55.934321 28530 raft_consensus.cc:2258] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:55.934573 28531 tablet_replica.cc:330] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: stopping tablet replica
I20260808 10:02:55.934674 28531 raft_consensus.cc:2229] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:55.934785 28531 raft_consensus.cc:2258] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:55.934952 28530 ts_tablet_manager.cc:1905] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:55.935236 28531 ts_tablet_manager.cc:1905] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:55.935412 28148 tablet_service.cc:1467] Processing CreateTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 (DEFAULT_TABLE table=test-table [id=72f73075616949d0a1dcbb5748e274c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.935527 28148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f25ec4ee314480d924aef802c19e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.936880 28477 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Bootstrap starting.
I20260808 10:02:55.937023 28530 ts_tablet_manager.cc:1918] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:55.937026 28531 ts_tablet_manager.cc:1918] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:55.937093 28531 log.cc:1196] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/1e8a9a354cda4000a8becb7a6337f47c
I20260808 10:02:55.937199 28530 log.cc:1196] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/1e8a9a354cda4000a8becb7a6337f47c
I20260808 10:02:55.937295 28477 tablet_bootstrap.cc:654] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.937377 28531 ts_tablet_manager.cc:1939] T 1e8a9a354cda4000a8becb7a6337f47c P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting consensus metadata
I20260808 10:02:55.937676 28530 ts_tablet_manager.cc:1939] T 1e8a9a354cda4000a8becb7a6337f47c P a200102e32e040b2a43cdae4748cd19c: Deleting consensus metadata
I20260808 10:02:55.937960 27925 tablet_service.cc:1467] Processing CreateTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 (DEFAULT_TABLE table=test-table [id=72f73075616949d0a1dcbb5748e274c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.937999 28345 tablet_service.cc:1467] Processing CreateTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 (DEFAULT_TABLE table=test-table [id=72f73075616949d0a1dcbb5748e274c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:55.938078 27925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f25ec4ee314480d924aef802c19e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.938095 28345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f25ec4ee314480d924aef802c19e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:55.938490 27696 catalog_manager.cc:4841] TS a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183): tablet 1e8a9a354cda4000a8becb7a6337f47c (table test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]) successfully deleted
I20260808 10:02:55.939034 28528 tablet_replica.cc:330] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: stopping tablet replica
I20260808 10:02:55.939177 28528 raft_consensus.cc:2229] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:55.939260 28471 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Bootstrap starting.
I20260808 10:02:55.939287 28528 raft_consensus.cc:2258] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:55.939646 28471 tablet_bootstrap.cc:654] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.939956 28482 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Bootstrap starting.
I20260808 10:02:55.940301 28471 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: No bootstrap required, opened a new log
I20260808 10:02:55.940346 28482 tablet_bootstrap.cc:654] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:55.940372 28471 ts_tablet_manager.cc:1397] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:55.940543 28471 raft_consensus.cc:348] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.940655 28471 raft_consensus.cc:374] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.940686 28471 raft_consensus.cc:729] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.940725 28471 consensus_queue.cc:260] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.940794 27695 catalog_manager.cc:4841] TS 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413): tablet 1e8a9a354cda4000a8becb7a6337f47c (table test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]) successfully deleted
I20260808 10:02:55.940836 28471 ts_tablet_manager.cc:1428] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.941337 28482 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: No bootstrap required, opened a new log
I20260808 10:02:55.941388 28482 ts_tablet_manager.cc:1397] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:55.941527 28482 raft_consensus.cc:348] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.941586 28482 raft_consensus.cc:374] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.941604 28482 raft_consensus.cc:729] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954c8e3de8949a7ad55fe0111f45911, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.941642 28482 consensus_queue.cc:260] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.941728 28482 ts_tablet_manager.cc:1428] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:55.941761 28528 ts_tablet_manager.cc:1905] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:55.942468 28477 tablet_bootstrap.cc:492] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: No bootstrap required, opened a new log
I20260808 10:02:55.942518 28477 ts_tablet_manager.cc:1397] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 10:02:55.942663 28477 raft_consensus.cc:348] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.942734 28477 raft_consensus.cc:374] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:55.942755 28477 raft_consensus.cc:729] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Initialized, Role: FOLLOWER
I20260808 10:02:55.942804 28477 consensus_queue.cc:260] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.942898 28477 ts_tablet_manager.cc:1428] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:55.943428 28528 ts_tablet_manager.cc:1918] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:55.943502 28528 log.cc:1196] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/1e8a9a354cda4000a8becb7a6337f47c
I20260808 10:02:55.943745 28528 ts_tablet_manager.cc:1939] T 1e8a9a354cda4000a8becb7a6337f47c P 9954c8e3de8949a7ad55fe0111f45911: Deleting consensus metadata
I20260808 10:02:55.944162 27695 catalog_manager.cc:4841] TS 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617): tablet 1e8a9a354cda4000a8becb7a6337f47c (table test-table [id=a0775bbd80464ae78b112c6f2c4bbc1c]) successfully deleted
W20260808 10:02:55.966574 28440 tablet.cc:2369] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:02:55.979044 28524 raft_consensus.cc:482] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:55.979126 28524 raft_consensus.cc:504] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.979271 28524 leader_election.cc:288] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183)
I20260808 10:02:55.979503 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f25ec4ee314480d924aef802c19e3f2" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c" is_pre_election: true
I20260808 10:02:55.979518 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f25ec4ee314480d924aef802c19e3f2" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911" is_pre_election: true
I20260808 10:02:55.979610 28178 raft_consensus.cc:2454] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.979624 27956 raft_consensus.cc:2454] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 0.
I20260808 10:02:55.979773 28285 leader_election.cc:302] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:55.979863 28524 raft_consensus.cc:2781] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:55.979902 28524 raft_consensus.cc:482] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:55.979933 28524 raft_consensus.cc:3037] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.980656 28524 raft_consensus.cc:504] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.980813 28524 leader_election.cc:288] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Requested vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183)
I20260808 10:02:55.980981 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f25ec4ee314480d924aef802c19e3f2" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911"
I20260808 10:02:55.981065 27956 raft_consensus.cc:3037] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.981015 28178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f25ec4ee314480d924aef802c19e3f2" candidate_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a200102e32e040b2a43cdae4748cd19c"
I20260808 10:02:55.981110 28178 raft_consensus.cc:3037] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:55.981703 28178 raft_consensus.cc:2454] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.981882 28285 leader_election.cc:302] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:55.981946 27956 raft_consensus.cc:2454] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe5a15d6da648b2b36e3ecc188d6cb6 in term 1.
I20260808 10:02:55.981992 28524 raft_consensus.cc:2781] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:55.982040 28524 raft_consensus.cc:686] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Running, Role: LEADER
I20260808 10:02:55.982095 28524 consensus_queue.cc:237] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:55.982730 27716 catalog_manager.cc:5495] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } health_report { overall_health: HEALTHY } } }
I20260808 10:02:55.988759 27956 raft_consensus.cc:1261] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.988870 28178 raft_consensus.cc:1261] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Refusing update from remote peer 1fe5a15d6da648b2b36e3ecc188d6cb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:55.988972 28489 consensus_queue.cc:1035] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:55.989197 28489 consensus_queue.cc:1035] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:02:55.993955 28506 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:02:57.517285 27714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:57.517457 27714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:57.518463 27714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Sending DeleteTablet for 3 replicas of tablet 8f25ec4ee314480d924aef802c19e3f2
I20260808 10:02:57.518743 28148 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:34710
I20260808 10:02:57.518759 27925 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46482
I20260808 10:02:57.518982 28344 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f25ec4ee314480d924aef802c19e3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:51548
I20260808 10:02:57.519057 28580 tablet_replica.cc:330] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: stopping tablet replica
I20260808 10:02:57.519114 28580 raft_consensus.cc:2229] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:57.519155 28580 raft_consensus.cc:2258] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.519222 28581 tablet_replica.cc:330] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: stopping tablet replica
I20260808 10:02:57.519294 28576 tablet_replica.cc:330] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: stopping tablet replica
I20260808 10:02:57.519353 28581 raft_consensus.cc:2229] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:57.519438 28576 raft_consensus.cc:2229] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:57.519447 28581 raft_consensus.cc:2258] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.519455 28580 ts_tablet_manager.cc:1905] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.519555 28576 raft_consensus.cc:2258] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.519695 28581 ts_tablet_manager.cc:1905] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.519874 28576 ts_tablet_manager.cc:1905] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.521653 28576 ts_tablet_manager.cc:1918] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:02:57.521785 28576 log.cc:1196] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/8f25ec4ee314480d924aef802c19e3f2
I20260808 10:02:57.521780 28581 ts_tablet_manager.cc:1918] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:02:57.521849 28581 log.cc:1196] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/8f25ec4ee314480d924aef802c19e3f2
I20260808 10:02:57.522178 28576 ts_tablet_manager.cc:1939] T 8f25ec4ee314480d924aef802c19e3f2 P 9954c8e3de8949a7ad55fe0111f45911: Deleting consensus metadata
I20260808 10:02:57.522205 28581 ts_tablet_manager.cc:1939] T 8f25ec4ee314480d924aef802c19e3f2 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting consensus metadata
I20260808 10:02:57.522218 28580 ts_tablet_manager.cc:1918] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:02:57.522274 28580 log.cc:1196] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/8f25ec4ee314480d924aef802c19e3f2
I20260808 10:02:57.522534 28580 ts_tablet_manager.cc:1939] T 8f25ec4ee314480d924aef802c19e3f2 P a200102e32e040b2a43cdae4748cd19c: Deleting consensus metadata
I20260808 10:02:57.522604 27695 catalog_manager.cc:4841] TS 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413): tablet 8f25ec4ee314480d924aef802c19e3f2 (table test-table [id=72f73075616949d0a1dcbb5748e274c5]) successfully deleted
I20260808 10:02:57.522696 27695 catalog_manager.cc:4841] TS 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617): tablet 8f25ec4ee314480d924aef802c19e3f2 (table test-table [id=72f73075616949d0a1dcbb5748e274c5]) successfully deleted
I20260808 10:02:57.522899 27696 catalog_manager.cc:4841] TS a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183): tablet 8f25ec4ee314480d924aef802c19e3f2 (table test-table [id=72f73075616949d0a1dcbb5748e274c5]) successfully deleted
Aug 08 10:02:57 dist-test-slave-nfb5 krb5kdc[27647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:02:57 dist-test-slave-nfb5 krb5kdc[27647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:02:57.544756 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.537103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.24.162.190:33629)
0808 10:02:57.537272 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.537274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.537283 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:02:57.537321 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.537322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.537333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.537412 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:02:57.538054 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.538587 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.539484 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.539697 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.539776 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.542015 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:02:57.542017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:02:57.542018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:02:57.542025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:02:57.543525 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:57.543881 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:57.543882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:57.543883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:57.543944 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:02:57.544145 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:02:57.544146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:02:57.544146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:02:57.544264 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:02:57.544415 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.544536 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.544621 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260808 10:02:57.546343 27714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:02:57.546489 27714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:57.548028 27925 tablet_service.cc:1467] Processing CreateTablet for tablet e13430e99ce54c91b151a82b3bc3f831 (DEFAULT_TABLE table=test-table [id=cd46d3d747494c7a9359627bceda5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.548041 28344 tablet_service.cc:1467] Processing CreateTablet for tablet e13430e99ce54c91b151a82b3bc3f831 (DEFAULT_TABLE table=test-table [id=cd46d3d747494c7a9359627bceda5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.548197 27925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13430e99ce54c91b151a82b3bc3f831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.548197 28344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13430e99ce54c91b151a82b3bc3f831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.549226 28594 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Bootstrap starting.
I20260808 10:02:57.549275 28148 tablet_service.cc:1467] Processing CreateTablet for tablet e13430e99ce54c91b151a82b3bc3f831 (DEFAULT_TABLE table=test-table [id=cd46d3d747494c7a9359627bceda5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.549408 28148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e13430e99ce54c91b151a82b3bc3f831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.549465 28595 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Bootstrap starting.
I20260808 10:02:57.549654 28594 tablet_bootstrap.cc:654] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.549913 28595 tablet_bootstrap.cc:654] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.550308 28594 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: No bootstrap required, opened a new log
I20260808 10:02:57.550344 28594 ts_tablet_manager.cc:1397] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:02:57.550454 28594 raft_consensus.cc:348] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.550509 28594 raft_consensus.cc:374] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.550522 28594 raft_consensus.cc:729] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954c8e3de8949a7ad55fe0111f45911, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.550631 28598 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Bootstrap starting.
I20260808 10:02:57.550611 28594 consensus_queue.cc:260] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.550828 28594 ts_tablet_manager.cc:1428] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:57.551115 28598 tablet_bootstrap.cc:654] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.551332 28595 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: No bootstrap required, opened a new log
I20260808 10:02:57.551396 28595 ts_tablet_manager.cc:1397] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:02:57.551535 28595 raft_consensus.cc:348] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.551641 28595 raft_consensus.cc:374] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.551678 28595 raft_consensus.cc:729] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.551801 28598 tablet_bootstrap.cc:492] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: No bootstrap required, opened a new log
I20260808 10:02:57.551772 28595 consensus_queue.cc:260] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.551873 28598 ts_tablet_manager.cc:1397] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:02:57.551913 28595 ts_tablet_manager.cc:1428] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:02:57.552009 28598 raft_consensus.cc:348] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.552093 28598 raft_consensus.cc:374] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.552116 28598 raft_consensus.cc:729] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.552175 28598 consensus_queue.cc:260] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.552381 28598 ts_tablet_manager.cc:1428] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:57.692682 28603 raft_consensus.cc:482] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:57.692854 28603 raft_consensus.cc:504] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.693297 28603 leader_election.cc:288] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413)
I20260808 10:02:57.696738 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.693815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:50197 (local address 127.24.162.131:41413)
0808 10:02:57.693985 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.693987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.693993 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:02:57.694013 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.694014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.694021 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.694075 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:57.694249 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.694764 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.695964 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.696365 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.696423 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.696430 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.696607 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.696656 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260808 10:02:57.696733 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.693737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:41413 (local address 127.24.162.130:50197)
0808 10:02:57.693848 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:57.693857 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:57.693897 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:57.694106 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:57.694108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:57.694110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:57.694172 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.694173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.694865 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.694867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.695826 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.695829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.696455 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.696456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.696531 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.696536 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:02:57.696590 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:02:57.696631 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":34,"thread_start_us":71,"threads_started":1}
I20260808 10:02:57.696812 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.693552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:34313 (local address 127.24.162.129:45617)
0808 10:02:57.693749 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.693752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.693764 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:02:57.693794 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.693795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.693807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.693884 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:02:57.694028 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.695086 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.696037 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.696433 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.696510 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.696520 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.696680 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.696718 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
I20260808 10:02:57.696839 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.693487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45617 (local address 127.24.162.130:34313)
0808 10:02:57.693675 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:57.693688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:57.693739 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:57.693888 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:57.693889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:57.693896 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:02:57.693958 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.693960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.695174 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.695175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.695948 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.695951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.696530 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.696531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.696596 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.696601 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:02:57.696659 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:02:57.696696 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260808 10:02:57.697036 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13430e99ce54c91b151a82b3bc3f831" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911" is_pre_election: true
I20260808 10:02:57.697156 27956 raft_consensus.cc:2454] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 0.
I20260808 10:02:57.697248 28372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13430e99ce54c91b151a82b3bc3f831" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" is_pre_election: true
I20260808 10:02:57.697352 28372 raft_consensus.cc:2454] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 0.
I20260808 10:02:57.697363 28072 leader_election.cc:302] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9954c8e3de8949a7ad55fe0111f45911, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:57.697510 28603 raft_consensus.cc:2781] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:57.697562 28603 raft_consensus.cc:482] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:57.697584 28603 raft_consensus.cc:3037] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.698271 28603 raft_consensus.cc:504] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.698402 28603 leader_election.cc:288] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 election: Requested vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413)
I20260808 10:02:57.698567 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13430e99ce54c91b151a82b3bc3f831" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911"
I20260808 10:02:57.698602 28372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e13430e99ce54c91b151a82b3bc3f831" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6"
I20260808 10:02:57.698652 27956 raft_consensus.cc:3037] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.698666 28372 raft_consensus.cc:3037] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.699465 28372 raft_consensus.cc:2454] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 1.
I20260808 10:02:57.699465 27956 raft_consensus.cc:2454] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 1.
I20260808 10:02:57.699658 28072 leader_election.cc:302] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe5a15d6da648b2b36e3ecc188d6cb6, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:57.699766 28603 raft_consensus.cc:2781] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:57.699908 28603 raft_consensus.cc:686] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 1 LEADER]: Becoming Leader. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Running, Role: LEADER
I20260808 10:02:57.699965 28603 consensus_queue.cc:237] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.700733 27714 catalog_manager.cc:5495] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c reported cstate change: term changed from 0 to 1, leader changed from <none> to a200102e32e040b2a43cdae4748cd19c (127.24.162.130). New cstate: current_term: 1 leader_uuid: "a200102e32e040b2a43cdae4748cd19c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } health_report { overall_health: UNKNOWN } } }
I20260808 10:02:57.736528 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.733653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.24.162.130:33183)
0808 10:02:57.733884 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.733885 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.733891 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:02:57.733926 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.733926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.733946 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.734046 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:57.734212 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.734724 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.735582 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.735859 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.735961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.736148 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:57.736270 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:57.736350 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.736398 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.736422 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":136,"threads_started":1}
I20260808 10:02:57.741274 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.739033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:52785 (local address 127.24.162.131:41413)
0808 10:02:57.739091 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.739093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.739219 (+ 126us) server_negotiation.cc:407] Connection header received
0808 10:02:57.739257 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.739257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.739266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.739314 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:57.739415 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.739869 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.740591 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.740792 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.740845 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.741034 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:57.741066 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:57.741123 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.741198 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.741219 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:02:57.741292 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.738989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:41413 (local address 127.24.162.130:52785)
0808 10:02:57.739146 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:57.739156 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:57.739218 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:57.739316 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:57.739316 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:57.739318 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:57.739368 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.739369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.739940 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.739941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.740520 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.740521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.740864 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.740865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.740915 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.740928 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:57.741139 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:57.741141 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:02:57.741195 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:02:57.741229 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260808 10:02:57.743765 27956 raft_consensus.cc:1261] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Refusing update from remote peer a200102e32e040b2a43cdae4748cd19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:57.743811 28372 raft_consensus.cc:1261] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Refusing update from remote peer a200102e32e040b2a43cdae4748cd19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:57.744030 28603 consensus_queue.cc:1035] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:57.744205 28608 consensus_queue.cc:1035] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:57.751559 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.749104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:53593 (local address 127.24.162.130:33183)
0808 10:02:57.749141 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:02:57.749143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:02:57.749270 (+ 127us) server_negotiation.cc:407] Connection header received
0808 10:02:57.749355 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:02:57.749356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:02:57.749364 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:02:57.749419 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:02:57.749522 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.749998 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.750768 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.751073 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.751147 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.751302 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:02:57.751333 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:02:57.751410 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:02:57.751471 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:02:57.751491 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 10:02:57.751592 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:57.749056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:33183 (local address 127.24.162.131:53593)
0808 10:02:57.749187 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:57.749197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:02:57.749281 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:02:57.749423 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:02:57.749424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:02:57.749425 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:02:57.749472 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.749473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.750091 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.750093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.750673 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:02:57.750675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:02:57.751146 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:02:57.751147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:02:57.751205 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:02:57.751209 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:02:57.751412 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:02:57.751415 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:02:57.751466 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:02:57.751513 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260808 10:02:57.758245 27714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:57.758415 27714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:57.759110 27714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Sending DeleteTablet for 3 replicas of tablet e13430e99ce54c91b151a82b3bc3f831
I20260808 10:02:57.759409 28148 tablet_service.cc:1514] Processing DeleteTablet for tablet e13430e99ce54c91b151a82b3bc3f831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:34710
I20260808 10:02:57.759407 27925 tablet_service.cc:1514] Processing DeleteTablet for tablet e13430e99ce54c91b151a82b3bc3f831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46482
I20260808 10:02:57.759509 28580 tablet_replica.cc:330] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: stopping tablet replica
I20260808 10:02:57.759552 28580 raft_consensus.cc:2229] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:57.759542 28576 tablet_replica.cc:330] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: stopping tablet replica
I20260808 10:02:57.759613 28576 raft_consensus.cc:2229] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:57.759629 28580 raft_consensus.cc:2258] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.759655 28576 raft_consensus.cc:2258] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.759801 28344 tablet_service.cc:1514] Processing DeleteTablet for tablet e13430e99ce54c91b151a82b3bc3f831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:57 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:51548
I20260808 10:02:57.759776 27714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:02:57.759891 28581 tablet_replica.cc:330] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: stopping tablet replica
W20260808 10:02:57.759912 27714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:02:57.759944 28581 raft_consensus.cc:2229] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:57.759989 28581 raft_consensus.cc:2258] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:57.760293 28581 ts_tablet_manager.cc:1905] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.760562 28580 ts_tablet_manager.cc:1905] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.761215 28148 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 (DEFAULT_TABLE table=test-table [id=a8a3303e91a14e90abaee308b91f1b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.761420 28148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffe896ff5a14f01b4c03ca7cf540406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.762192 28581 ts_tablet_manager.cc:1918] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:57.762257 28581 log.cc:1196] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/e13430e99ce54c91b151a82b3bc3f831
I20260808 10:02:57.762267 28580 ts_tablet_manager.cc:1918] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:57.762315 28580 log.cc:1196] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/e13430e99ce54c91b151a82b3bc3f831
I20260808 10:02:57.762487 28581 ts_tablet_manager.cc:1939] T e13430e99ce54c91b151a82b3bc3f831 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting consensus metadata
I20260808 10:02:57.762540 28580 ts_tablet_manager.cc:1939] T e13430e99ce54c91b151a82b3bc3f831 P a200102e32e040b2a43cdae4748cd19c: Deleting consensus metadata
I20260808 10:02:57.762678 28598 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Bootstrap starting.
I20260808 10:02:57.762953 28576 ts_tablet_manager.cc:1905] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:57.763036 27925 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 (DEFAULT_TABLE table=test-table [id=a8a3303e91a14e90abaee308b91f1b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.763110 28598 tablet_bootstrap.cc:654] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.763140 27925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffe896ff5a14f01b4c03ca7cf540406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.764356 28594 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911: Bootstrap starting.
I20260808 10:02:57.764499 27696 catalog_manager.cc:4841] TS a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183): tablet e13430e99ce54c91b151a82b3bc3f831 (table test-table [id=cd46d3d747494c7a9359627bceda5bd5]) successfully deleted
I20260808 10:02:57.764819 28594 tablet_bootstrap.cc:654] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.764889 27695 catalog_manager.cc:4841] TS 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413): tablet e13430e99ce54c91b151a82b3bc3f831 (table test-table [id=cd46d3d747494c7a9359627bceda5bd5]) successfully deleted
I20260808 10:02:57.764901 28344 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 (DEFAULT_TABLE table=test-table [id=a8a3303e91a14e90abaee308b91f1b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:02:57.765003 28344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffe896ff5a14f01b4c03ca7cf540406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:02:57.764353 28576 ts_tablet_manager.cc:1918] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:02:57.765259 28576 log.cc:1196] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/e13430e99ce54c91b151a82b3bc3f831
I20260808 10:02:57.765523 28576 ts_tablet_manager.cc:1939] T e13430e99ce54c91b151a82b3bc3f831 P 9954c8e3de8949a7ad55fe0111f45911: Deleting consensus metadata
I20260808 10:02:57.765856 28598 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: No bootstrap required, opened a new log
I20260808 10:02:57.765903 28598 ts_tablet_manager.cc:1397] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:02:57.766074 28598 raft_consensus.cc:348] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.766139 28598 raft_consensus.cc:374] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.766130 28594 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911: No bootstrap required, opened a new log
I20260808 10:02:57.766158 28598 raft_consensus.cc:729] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.766173 28594 ts_tablet_manager.cc:1397] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:02:57.766208 28598 consensus_queue.cc:260] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.766321 28598 ts_tablet_manager.cc:1428] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:57.766304 28594 raft_consensus.cc:348] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.766383 28594 raft_consensus.cc:374] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.766405 28594 raft_consensus.cc:729] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954c8e3de8949a7ad55fe0111f45911, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.766415 27695 catalog_manager.cc:4841] TS 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617): tablet e13430e99ce54c91b151a82b3bc3f831 (table test-table [id=cd46d3d747494c7a9359627bceda5bd5]) successfully deleted
I20260808 10:02:57.766448 28594 consensus_queue.cc:260] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.766526 28594 ts_tablet_manager.cc:1428] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:02:57.766607 28595 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Bootstrap starting.
I20260808 10:02:57.766963 28595 tablet_bootstrap.cc:654] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Neither blocks nor log segments found. Creating new log.
I20260808 10:02:57.767570 28595 tablet_bootstrap.cc:492] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: No bootstrap required, opened a new log
I20260808 10:02:57.767617 28595 ts_tablet_manager.cc:1397] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:02:57.767712 28595 raft_consensus.cc:348] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.767774 28595 raft_consensus.cc:374] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:02:57.767795 28595 raft_consensus.cc:729] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe5a15d6da648b2b36e3ecc188d6cb6, State: Initialized, Role: FOLLOWER
I20260808 10:02:57.767844 28595 consensus_queue.cc:260] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.767935 28595 ts_tablet_manager.cc:1428] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:02:57.819088 28603 raft_consensus.cc:482] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:02:57.819200 28603 raft_consensus.cc:504] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.819388 28603 leader_election.cc:288] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413)
I20260808 10:02:57.819703 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffe896ff5a14f01b4c03ca7cf540406" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911" is_pre_election: true
I20260808 10:02:57.819736 28372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffe896ff5a14f01b4c03ca7cf540406" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" is_pre_election: true
I20260808 10:02:57.819835 28372 raft_consensus.cc:2454] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 0.
I20260808 10:02:57.819829 27956 raft_consensus.cc:2454] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 0.
I20260808 10:02:57.820022 28072 leader_election.cc:302] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9954c8e3de8949a7ad55fe0111f45911, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:57.820132 28603 raft_consensus.cc:2781] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:02:57.820174 28603 raft_consensus.cc:482] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:02:57.820199 28603 raft_consensus.cc:3037] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.820917 28603 raft_consensus.cc:504] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.821056 28603 leader_election.cc:288] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 election: Requested vote from peers 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617), 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413)
I20260808 10:02:57.821226 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffe896ff5a14f01b4c03ca7cf540406" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9954c8e3de8949a7ad55fe0111f45911"
I20260808 10:02:57.821226 28372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffe896ff5a14f01b4c03ca7cf540406" candidate_uuid: "a200102e32e040b2a43cdae4748cd19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6"
I20260808 10:02:57.821324 27956 raft_consensus.cc:3037] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.821336 28372 raft_consensus.cc:3037] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:02:57.821960 27956 raft_consensus.cc:2454] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 1.
I20260808 10:02:57.822150 28072 leader_election.cc:302] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9954c8e3de8949a7ad55fe0111f45911, a200102e32e040b2a43cdae4748cd19c; no voters:
I20260808 10:02:57.822217 28372 raft_consensus.cc:2454] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a200102e32e040b2a43cdae4748cd19c in term 1.
I20260808 10:02:57.822247 28603 raft_consensus.cc:2781] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:02:57.822289 28603 raft_consensus.cc:686] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 1 LEADER]: Becoming Leader. State: Replica: a200102e32e040b2a43cdae4748cd19c, State: Running, Role: LEADER
I20260808 10:02:57.822350 28603 consensus_queue.cc:237] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } }
I20260808 10:02:57.823031 27716 catalog_manager.cc:5495] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c reported cstate change: term changed from 0 to 1, leader changed from <none> to a200102e32e040b2a43cdae4748cd19c (127.24.162.130). New cstate: current_term: 1 leader_uuid: "a200102e32e040b2a43cdae4748cd19c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a200102e32e040b2a43cdae4748cd19c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 } health_report { overall_health: UNKNOWN } } }
I20260808 10:02:57.829383 28372 raft_consensus.cc:1261] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Refusing update from remote peer a200102e32e040b2a43cdae4748cd19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:57.829514 27956 raft_consensus.cc:1261] T 4ffe896ff5a14f01b4c03ca7cf540406 P 9954c8e3de8949a7ad55fe0111f45911 [term 1 FOLLOWER]: Refusing update from remote peer a200102e32e040b2a43cdae4748cd19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:02:57.829646 28603 consensus_queue.cc:1035] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe5a15d6da648b2b36e3ecc188d6cb6" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 41413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:02:57.829743 28608 consensus_queue.cc:1035] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9954c8e3de8949a7ad55fe0111f45911" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:02:57.834511 28567 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:02:59.353443 27714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:59.353557 27714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:02:59.354413 27714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90c5ef309f143b6ad96b566065a9c18: Sending DeleteTablet for 3 replicas of tablet 4ffe896ff5a14f01b4c03ca7cf540406
I20260808 10:02:59.354715 28344 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:59 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:51548
I20260808 10:02:59.354700 27925 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:59 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46482
I20260808 10:02:59.354866 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 27831
I20260808 10:02:59.354897 28148 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ffe896ff5a14f01b4c03ca7cf540406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:02:59 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:34710
I20260808 10:02:59.367385 28682 tablet_replica.cc:330] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: stopping tablet replica
I20260808 10:02:59.367455 28682 raft_consensus.cc:2229] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:02:59.367507 28682 raft_consensus.cc:2258] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:59.367633 28680 tablet_replica.cc:330] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: stopping tablet replica
I20260808 10:02:59.367698 28680 raft_consensus.cc:2229] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:02:59.367736 28680 raft_consensus.cc:2258] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:02:59.367950 28682 ts_tablet_manager.cc:1905] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:59.369433 28680 ts_tablet_manager.cc:1905] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:02:59.369717 28682 ts_tablet_manager.cc:1918] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:02:59.369768 28682 log.cc:1196] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/4ffe896ff5a14f01b4c03ca7cf540406
I20260808 10:02:59.370012 28682 ts_tablet_manager.cc:1939] T 4ffe896ff5a14f01b4c03ca7cf540406 P a200102e32e040b2a43cdae4748cd19c: Deleting consensus metadata
I20260808 10:02:59.370378 27696 catalog_manager.cc:4841] TS a200102e32e040b2a43cdae4748cd19c (127.24.162.130:33183): tablet 4ffe896ff5a14f01b4c03ca7cf540406 (table test-table [id=a8a3303e91a14e90abaee308b91f1b60]) successfully deleted
I20260808 10:02:59.371238 28680 ts_tablet_manager.cc:1918] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:02:59.371289 28680 log.cc:1196] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/4ffe896ff5a14f01b4c03ca7cf540406
I20260808 10:02:59.371498 28680 ts_tablet_manager.cc:1939] T 4ffe896ff5a14f01b4c03ca7cf540406 P 1fe5a15d6da648b2b36e3ecc188d6cb6: Deleting consensus metadata
I20260808 10:02:59.372150 27695 catalog_manager.cc:4841] TS 1fe5a15d6da648b2b36e3ecc188d6cb6 (127.24.162.131:41413): tablet 4ffe896ff5a14f01b4c03ca7cf540406 (table test-table [id=a8a3303e91a14e90abaee308b91f1b60]) successfully deleted
W20260808 10:02:59.376231 27695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:45617 (error 108)
I20260808 10:02:59.376447 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 28045
I20260808 10:02:59.380956 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:02:59.380653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45617 (local address 127.0.0.1:59608)
0808 10:02:59.380845 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:02:59.380902 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
W20260808 10:02:59.381106 27695 catalog_manager.cc:4584] TS 9954c8e3de8949a7ad55fe0111f45911 (127.24.162.129:45617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ffe896ff5a14f01b4c03ca7cf540406: Network error: Client connection negotiation failed: client connection to 127.24.162.129:45617: connect: Connection refused (error 111)
I20260808 10:02:59.387435 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 28254
I20260808 10:02:59.396337 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 27667
2026-08-08T10:02:59Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5649 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:02:59 dist-test-slave-nfb5 krb5kdc[28688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:02:59 dist-test-slave-nfb5 krb5kdc[28688](info): set up 2 sockets
Aug 08 10:02:59 dist-test-slave-nfb5 krb5kdc[28688](info): commencing operation
krb5kdc: starting...
W20260808 10:03:00.629381 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:00 dist-test-slave-nfb5 krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:00.758047 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36551
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:00.843277 28702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:00.843499 28702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:00.843534 28702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:00.844662 28702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:00.844731 28702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:00.844756 28702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:00.844779 28702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:00.844808 28702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:00.846277 28702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:00.846549 28702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:00.846779 28702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:00.852033 28702 server_base.cc:1034] running on GCE node
W20260808 10:03:00.852144 28713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:00.852144 28712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:00.852435 28715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:00.852679 28702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:00.853030 28702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:00.854190 28702 hybrid_clock.cc:648] HybridClock initialized: now 1786183380854140 us; error 69 us; skew 500 ppm
Aug 08 10:03:00 dist-test-slave-nfb5 krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:00.856702 28702 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:00.857184 28702 webserver.cc:460] Webserver started at http://127.24.162.190:35227/ using document root <none> and password file <none>
I20260808 10:03:00.857445 28702 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:00.857515 28702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:00.857664 28702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:00.858707 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "88445d68a60f452daf8507ee9c93625d"
format_stamp: "Formatted at 2026-08-08 10:03:00 on dist-test-slave-nfb5"
server_key: "eaf35d32fc1f4538407f640904c1e07e"
server_key_iv: "cb329f033ffb4abb719847b9f4ec75b2"
server_key_version: "encryptionkey@0"
I20260808 10:03:00.859107 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "88445d68a60f452daf8507ee9c93625d"
format_stamp: "Formatted at 2026-08-08 10:03:00 on dist-test-slave-nfb5"
server_key: "eaf35d32fc1f4538407f640904c1e07e"
server_key_iv: "cb329f033ffb4abb719847b9f4ec75b2"
server_key_version: "encryptionkey@0"
I20260808 10:03:00.860469 28702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:00.861240 28725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:00.861425 28702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:00.861528 28702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "88445d68a60f452daf8507ee9c93625d"
format_stamp: "Formatted at 2026-08-08 10:03:00 on dist-test-slave-nfb5"
server_key: "eaf35d32fc1f4538407f640904c1e07e"
server_key_iv: "cb329f033ffb4abb719847b9f4ec75b2"
server_key_version: "encryptionkey@0"
I20260808 10:03:00.861608 28702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:00.877359 28702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:00.879385 28702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:00.879544 28702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:00.884250 28702 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36551
I20260808 10:03:00.884289 28797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36551 every 8 connection(s)
I20260808 10:03:00.884634 28702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:00.885541 28798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:00.887776 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d: Bootstrap starting.
I20260808 10:03:00.888317 28798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:00.888558 28798 log.cc:824] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:00.889271 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d: No bootstrap required, opened a new log
I20260808 10:03:00.890828 28798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } }
I20260808 10:03:00.891036 28798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:00.891065 28798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88445d68a60f452daf8507ee9c93625d, State: Initialized, Role: FOLLOWER
I20260808 10:03:00.891173 28798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } }
I20260808 10:03:00.891233 28798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:00.891256 28798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:00.891289 28798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:00.891803 28798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } }
I20260808 10:03:00.891889 28798 leader_election.cc:302] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88445d68a60f452daf8507ee9c93625d; no voters:
I20260808 10:03:00.892011 28798 leader_election.cc:288] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:00.892062 28801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:00.892233 28801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [term 1 LEADER]: Becoming Leader. State: Replica: 88445d68a60f452daf8507ee9c93625d, State: Running, Role: LEADER
I20260808 10:03:00.892282 28798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:00.892364 28801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } }
I20260808 10:03:00.892817 28803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88445d68a60f452daf8507ee9c93625d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } } }
I20260808 10:03:00.892818 28806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88445d68a60f452daf8507ee9c93625d. Latest consensus state: current_term: 1 leader_uuid: "88445d68a60f452daf8507ee9c93625d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445d68a60f452daf8507ee9c93625d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36551 } } }
I20260808 10:03:00.893016 28806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:00.893245 28803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:00.893504 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 28702
I20260808 10:03:00.893599 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:00.893778 25226 external_mini_cluster.cc:1428] Setting key c0d97718d6356f126a554e232eebca54
I20260808 10:03:00.894073 28810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:00.894582 28810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:00.896878 28810 catalog_manager.cc:1347] Generated new cluster ID: dd8a17a3226d4d60bfb820385524c5b9
I20260808 10:03:00.896937 28810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:00 dist-test-slave-nfb5 krb5kdc[28688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:00.905915 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:00.894244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.24.162.190:36551)
0808 10:03:00.895066 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:00.895075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:00.895286 (+ 211us) server_negotiation.cc:407] Connection header received
0808 10:03:00.895416 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:00.895422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:00.895477 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:00.895630 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:00.896279 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:00.897620 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:00.898660 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:00.899258 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:00.899352 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:00.902144 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:00.902148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:00.902149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:00.902159 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:00.904152 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:00.904522 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:00.904525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:00.904525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:00.904615 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:00.904778 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:00.904779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:00.904779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:00.905056 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:00.905174 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:00.905567 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 10:03:00.905690 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":85,"threads_started":1}
I20260808 10:03:00.914873 28810 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:00.915391 28810 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:00.920092 28810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88445d68a60f452daf8507ee9c93625d: Generated new TSK 0
I20260808 10:03:00.920260 28810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:00.951747 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:01.033356 28846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:01.033553 28846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:01.033576 28846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:01.034641 28846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:01.034674 28846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:01.034699 28846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:01.036265 28846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:01.036455 28846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:01.036624 28846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:01.041427 28857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:01.041633 28846 server_base.cc:1034] running on GCE node
W20260808 10:03:01.041435 28855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:01.041435 28854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:01.041890 28846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:01.042140 28846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:01.043304 28846 hybrid_clock.cc:648] HybridClock initialized: now 1786183381043286 us; error 41 us; skew 500 ppm
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:01.045759 28846 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:01.046231 28846 webserver.cc:460] Webserver started at http://127.24.162.129:46867/ using document root <none> and password file <none>
I20260808 10:03:01.046473 28846 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:01.046543 28846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:01.046692 28846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:01.047799 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "90f0769c2a4d4095ba91f7682bac585b"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "1eb664260e59dd2114960b782dca966f"
server_key_iv: "d559ae225d04f3169a767413178d101e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.048164 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "90f0769c2a4d4095ba91f7682bac585b"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "1eb664260e59dd2114960b782dca966f"
server_key_iv: "d559ae225d04f3169a767413178d101e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.049489 28846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:03:01.050220 28867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.050401 28846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:01.050501 28846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "90f0769c2a4d4095ba91f7682bac585b"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "1eb664260e59dd2114960b782dca966f"
server_key_iv: "d559ae225d04f3169a767413178d101e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.050588 28846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:01.063953 28846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:01.065956 28846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:01.066119 28846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:01.066351 28846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:01.066664 28846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:01.066720 28846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.066756 28846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:01.066782 28846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.072865 28846 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:46051
I20260808 10:03:01.072904 29027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:46051 every 8 connection(s)
I20260808 10:03:01.073230 28846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:01.076452 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 28846
I20260808 10:03:01.076525 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:01.076742 25226 external_mini_cluster.cc:1428] Setting key 349c4e0c2473f70b3ebc215207e0bc45
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:01.084460 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.074091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:47481 (local address 127.24.162.190:36551)
0808 10:03:01.074185 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:01.074188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:01.074351 (+ 163us) server_negotiation.cc:407] Connection header received
0808 10:03:01.074884 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:01.074885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:01.074901 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:01.074996 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:01.075960 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.076473 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.077974 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.078172 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.078230 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.080929 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:01.080931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:01.080931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:01.080939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:01.082797 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.083389 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.083390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.083390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.083445 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.083766 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.083766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.083767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.083880 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:01.083926 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:01.084309 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 10:03:01.084391 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:03:01.084724 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.074041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36551 (local address 127.24.162.129:47481)
0808 10:03:01.074236 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:01.074274 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:01.074794 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:01.075057 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:01.075060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:01.075631 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:01.075826 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.075830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.076626 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.076628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.077874 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.077877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.078261 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.078262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.078416 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.078965 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:01.078975 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:01.080825 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:01.082889 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.082898 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.082900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.083308 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.083529 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.083530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.083531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.083689 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.083946 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:01.083955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:01.084140 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 10:03:01.084280 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 10:03:01.084464 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260808 10:03:01.085290 29028 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36551
I20260808 10:03:01.085410 29028 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:01.085615 29028 heartbeater.cc:511] Master 127.24.162.190:36551 requested a full tablet report, sending...
I20260808 10:03:01.086171 28750 ts_manager.cc:194] Registered new tserver with Master: 90f0769c2a4d4095ba91f7682bac585b (127.24.162.129:46051)
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
I20260808 10:03:01.086783 28750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:47481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:01.104007 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:01.185726 29034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:01.185923 29034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:01.185946 29034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:01.187160 29034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:01.187203 29034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:01.187229 29034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:01.188697 29034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:01.188951 29034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:01.189191 29034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:01.194307 29040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:01.194327 29041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:01.194613 29034 server_base.cc:1034] running on GCE node
W20260808 10:03:01.194761 29045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:01.195039 29034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:01.195266 29034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:01.196415 29034 hybrid_clock.cc:648] HybridClock initialized: now 1786183381196400 us; error 31 us; skew 500 ppm
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:01.199162 29034 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:01.199653 29034 webserver.cc:460] Webserver started at http://127.24.162.130:33401/ using document root <none> and password file <none>
I20260808 10:03:01.199918 29034 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:01.199995 29034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:01.200160 29034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:01.201318 29034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "849e11270f4d4ccca5239fc582058267"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "c41d36b05e6b36022cbc45402892577e"
server_key_iv: "afff3e59a1590e069750d7ddc164e845"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.201725 29034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "849e11270f4d4ccca5239fc582058267"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "c41d36b05e6b36022cbc45402892577e"
server_key_iv: "afff3e59a1590e069750d7ddc164e845"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.203248 29034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:01.204041 29054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.204254 29034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:01.204375 29034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "849e11270f4d4ccca5239fc582058267"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "c41d36b05e6b36022cbc45402892577e"
server_key_iv: "afff3e59a1590e069750d7ddc164e845"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.204455 29034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:01.215391 29034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:01.217301 29034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:01.217492 29034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:01.217777 29034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:01.218137 29034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:01.218237 29034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.218303 29034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:01.218338 29034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.225163 29034 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:46057
I20260808 10:03:01.225217 29223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:46057 every 8 connection(s)
I20260808 10:03:01.225553 29034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:01.228888 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29034
I20260808 10:03:01.228976 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:01.229183 25226 external_mini_cluster.cc:1428] Setting key ee371c9a74411c2806966f6a02b87d54
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:01.235420 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.226264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:46767 (local address 127.24.162.190:36551)
0808 10:03:01.226344 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:01.226347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:01.226573 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:03:01.227152 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:01.227154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:01.227166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:01.227230 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:01.228161 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.228667 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.230074 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.230279 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.230368 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.232555 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:01.232556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:01.232557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:01.232565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:01.233914 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.234410 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.234411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.234411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.234474 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.234735 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.234736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.234737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.234861 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:01.234958 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:01.235237 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 10:03:01.235330 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:03:01.235618 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.226251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36551 (local address 127.24.162.130:46767)
0808 10:03:01.226453 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:01.226487 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:01.227046 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:01.227298 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:01.227302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:01.227873 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:01.228059 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.228064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.228765 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.228766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.229963 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.229966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.230378 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.230379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.230521 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.231102 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:01.231110 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:01.232456 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:01.234100 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.234107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.234108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.234325 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.234545 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.234546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.234546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.234663 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.234950 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:01.234958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:01.235097 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 10:03:01.235203 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:03:01.235374 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260808 10:03:01.236122 29224 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36551
I20260808 10:03:01.236222 29224 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:01.236459 29224 heartbeater.cc:511] Master 127.24.162.190:36551 requested a full tablet report, sending...
I20260808 10:03:01.236891 28750 ts_manager.cc:194] Registered new tserver with Master: 849e11270f4d4ccca5239fc582058267 (127.24.162.130:46057)
I20260808 10:03:01.237262 28750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:46767
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:01.257416 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:01.339710 29232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:01.339957 29232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:01.339998 29232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:01.341167 29232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:01.341238 29232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:01.341284 29232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:01.342700 29232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:01.343012 29232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:01.343251 29232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:01.348224 29240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:01.348208 29242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:01.348232 29239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:01.348482 29232 server_base.cc:1034] running on GCE node
I20260808 10:03:01.348719 29232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:01.348963 29232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:01.350123 29232 hybrid_clock.cc:648] HybridClock initialized: now 1786183381350098 us; error 38 us; skew 500 ppm
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:01.352697 29232 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:01.353159 29232 webserver.cc:460] Webserver started at http://127.24.162.131:42497/ using document root <none> and password file <none>
I20260808 10:03:01.353408 29232 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:01.353479 29232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:01.353621 29232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:01.354635 29232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "99dfa72e4f0a4a6aa7f4c1f010ad822a"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "d3d93582bf83669cb1dfbc361c76ac87"
server_key_iv: "8286ecda1eb766a0f8f8f7d42b49c38e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.355057 29232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "99dfa72e4f0a4a6aa7f4c1f010ad822a"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "d3d93582bf83669cb1dfbc361c76ac87"
server_key_iv: "8286ecda1eb766a0f8f8f7d42b49c38e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.356760 29232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:01.357535 29251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.357725 29232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:01.357827 29232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "99dfa72e4f0a4a6aa7f4c1f010ad822a"
format_stamp: "Formatted at 2026-08-08 10:03:01 on dist-test-slave-nfb5"
server_key: "d3d93582bf83669cb1dfbc361c76ac87"
server_key_iv: "8286ecda1eb766a0f8f8f7d42b49c38e"
server_key_version: "encryptionkey@0"
I20260808 10:03:01.357900 29232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:01.369616 29232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:01.371675 29232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:01.371837 29232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:01.372067 29232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:01.372381 29232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:01.372438 29232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.372480 29232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:01.372509 29232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:01.378211 29232 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:45985
I20260808 10:03:01.378283 29401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:45985 every 8 connection(s)
I20260808 10:03:01.378607 29232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:01.382357 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29232
I20260808 10:03:01.382458 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:01.382653 25226 external_mini_cluster.cc:1428] Setting key f9f31fa895a94cb69bf5961c365c86ad
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[28688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183381, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:01.388522 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.379381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:57201 (local address 127.24.162.190:36551)
0808 10:03:01.379451 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:01.379453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:01.379710 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:03:01.380168 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:01.380169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:01.380179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:01.380227 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:01.381255 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.381972 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.383518 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.383751 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.383816 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.385908 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:01.385910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:01.385910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:01.385916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:01.387391 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.387744 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.387745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.387745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.387796 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:01.388021 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:01.388022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:01.388022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:01.388120 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:01.388170 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:01.388394 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 10:03:01.388458 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:03:01.388681 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:01.379417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36551 (local address 127.24.162.131:57201)
0808 10:03:01.379591 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:01.379626 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:01.380085 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:01.380287 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:01.380290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:01.380874 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:01.381154 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.381159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.382085 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.382086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.383432 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:01.383436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:01.383830 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:01.383830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:01.383932 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:01.384432 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:01.384440 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:01.385812 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:01.387459 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.387464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.387466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.387678 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.387870 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:01.387871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:01.387871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:01.387966 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:01.388173 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:01.388177 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:01.388282 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 10:03:01.388366 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:03:01.388481 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260808 10:03:01.389243 29403 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36551
I20260808 10:03:01.389350 29403 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:01.389528 29403 heartbeater.cc:511] Master 127.24.162.190:36551 requested a full tablet report, sending...
I20260808 10:03:01.389921 28750 ts_manager.cc:194] Registered new tserver with Master: 99dfa72e4f0a4a6aa7f4c1f010ad822a (127.24.162.131:45985)
I20260808 10:03:01.390290 28750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:57201
I20260808 10:03:01.394309 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:03:01.406296 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:01.405356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.24.162.190:36551)
0808 10:03:01.405424 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:01.405426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:01.405523 (+ 97us) server_negotiation.cc:407] Connection header received
0808 10:03:01.405635 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:01.405636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:01.405648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:01.405717 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:01.406245 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54690: BlockingRecv error: recv got EOF from 127.0.0.1:54690 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:03:01.407343 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 28846
I20260808 10:03:01.421572 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29034
I20260808 10:03:01.431193 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29232
I20260808 10:03:01.453619 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 28702
2026-08-08T10:03:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2053 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[29419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[29419](info): set up 2 sockets
Aug 08 10:03:01 dist-test-slave-nfb5 krb5kdc[29419](info): commencing operation
krb5kdc: starting...
W20260808 10:03:02.685235 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:02 dist-test-slave-nfb5 krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:02.809381 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:42855
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:02.891058 29436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:02.891258 29436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:02.891283 29436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:02.892455 29436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:02.892498 29436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:02.892513 29436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:02.892526 29436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:02.892580 29436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:02.894066 29436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:42855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:02.894326 29436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:02.894564 29436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:02.899339 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:02.899353 29443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:02.899360 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:02.899619 29436 server_base.cc:1034] running on GCE node
I20260808 10:03:02.899873 29436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:02.900156 29436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:02.901297 29436 hybrid_clock.cc:648] HybridClock initialized: now 1786183382901273 us; error 36 us; skew 500 ppm
Aug 08 10:03:02 dist-test-slave-nfb5 krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183382, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:02.904011 29436 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:02.904480 29436 webserver.cc:460] Webserver started at http://127.24.162.190:43409/ using document root <none> and password file <none>
I20260808 10:03:02.904699 29436 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:02.904743 29436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:02.904910 29436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:02.905939 29436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "64e08de94adf48e7b702d195a529c469"
format_stamp: "Formatted at 2026-08-08 10:03:02 on dist-test-slave-nfb5"
server_key: "e53658a3a2d04f6b0b7eaedca7c33784"
server_key_iv: "d5679a49aea0d7ebc35e8e16ce22ce9a"
server_key_version: "encryptionkey@0"
I20260808 10:03:02.906302 29436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "64e08de94adf48e7b702d195a529c469"
format_stamp: "Formatted at 2026-08-08 10:03:02 on dist-test-slave-nfb5"
server_key: "e53658a3a2d04f6b0b7eaedca7c33784"
server_key_iv: "d5679a49aea0d7ebc35e8e16ce22ce9a"
server_key_version: "encryptionkey@0"
I20260808 10:03:02.907733 29436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:02.908540 29456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:02.908775 29436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:02.908879 29436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "64e08de94adf48e7b702d195a529c469"
format_stamp: "Formatted at 2026-08-08 10:03:02 on dist-test-slave-nfb5"
server_key: "e53658a3a2d04f6b0b7eaedca7c33784"
server_key_iv: "d5679a49aea0d7ebc35e8e16ce22ce9a"
server_key_version: "encryptionkey@0"
I20260808 10:03:02.908951 29436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:02.924867 29436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:02.927368 29436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:02.927547 29436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:02.931967 29436 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:42855
I20260808 10:03:02.932006 29527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:42855 every 8 connection(s)
I20260808 10:03:02.932587 29436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:02.933331 29529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:02.934320 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29436
I20260808 10:03:02.934427 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:02.934616 25226 external_mini_cluster.cc:1428] Setting key cf1c728988fa6541215484f68de91dae
I20260808 10:03:02.937054 29529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469: Bootstrap starting.
I20260808 10:03:02.937723 29529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:02.938032 29529 log.cc:824] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:02.938817 29529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469: No bootstrap required, opened a new log
I20260808 10:03:02.941483 29529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } }
I20260808 10:03:02.941618 29529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:02.941639 29529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e08de94adf48e7b702d195a529c469, State: Initialized, Role: FOLLOWER
I20260808 10:03:02.941773 29529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } }
I20260808 10:03:02.941848 29529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 08 10:03:02 dist-test-slave-nfb5 krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:02.941888 29529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:02.941932 29529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:02.942582 29529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } }
I20260808 10:03:02.942701 29529 leader_election.cc:302] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e08de94adf48e7b702d195a529c469; no voters:
I20260808 10:03:02.942860 29529 leader_election.cc:288] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:02.943101 29537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:02.943135 29529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:02.943231 29537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [term 1 LEADER]: Becoming Leader. State: Replica: 64e08de94adf48e7b702d195a529c469, State: Running, Role: LEADER
I20260808 10:03:02.943342 29537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } }
I20260808 10:03:02.943706 29540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e08de94adf48e7b702d195a529c469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } } }
I20260808 10:03:02.943811 29540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:02.944005 29541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e08de94adf48e7b702d195a529c469. Latest consensus state: current_term: 1 leader_uuid: "64e08de94adf48e7b702d195a529c469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e08de94adf48e7b702d195a529c469" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42855 } } }
I20260808 10:03:02.944046 29554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:02.944057 29541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:02.944602 29554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:02.946287 29554 catalog_manager.cc:1347] Generated new cluster ID: 4f68a911d61e4829965b429f7a1ff113
I20260808 10:03:02.946348 29554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:02.946493 29535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:02.935259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.24.162.190:42855)
0808 10:03:02.935458 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:02.935467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:02.935494 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:03:02.935925 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:02.935931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:02.935977 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:02.936090 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:02.937036 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:02.938036 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:02.939047 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:02.939444 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:02.939526 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:02.942364 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:02.942369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:02.942370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:02.942384 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:02.944763 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:02.945198 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:02.945201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:02.945201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:02.945315 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:02.945555 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:02.945556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:02.945556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:02.945866 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:02.945998 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:02.946124 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 10:03:02.946264 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260808 10:03:02.947863 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:03:02.954988 29554 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:02.955497 29554 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:02.960695 29554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e08de94adf48e7b702d195a529c469: Generated new TSK 0
I20260808 10:03:02.960884 29554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:02.966306 29535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:02.962556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.24.162.190:42855)
0808 10:03:02.962601 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:02.962604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:02.962725 (+ 121us) server_negotiation.cc:407] Connection header received
0808 10:03:02.962792 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:02.962793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:02.962801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:02.962852 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:02.963294 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:02.963770 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:02.964901 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:02.965097 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:02.965157 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:02.965838 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:02.965839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:02.965839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:02.966023 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:02.966038 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:02.966044 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:02.966091 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:02.966181 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 10:03:02.966208 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:03:02.968771 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29436
2026-08-08T10:03:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1521 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:03 dist-test-slave-nfb5 krb5kdc[29577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:03 dist-test-slave-nfb5 krb5kdc[29577](info): set up 2 sockets
Aug 08 10:03:03 dist-test-slave-nfb5 krb5kdc[29577](info): commencing operation
krb5kdc: starting...
W20260808 10:03:04.200088 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:04.322093 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:39943
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:39943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:04.403968 29596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:04.404218 29596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:04.404259 29596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:04.405439 29596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:04.405505 29596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:04.405529 29596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:04.405552 29596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:04.405579 29596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:04.407140 29596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:39943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:39943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:04.407411 29596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:04.407636 29596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:04.412624 29596 server_base.cc:1034] running on GCE node
W20260808 10:03:04.412606 29601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.412599 29602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.412607 29606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:04.413117 29596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:04.413457 29596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:04.414649 29596 hybrid_clock.cc:648] HybridClock initialized: now 1786183384414632 us; error 27 us; skew 500 ppm
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:04.417292 29596 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:04.417784 29596 webserver.cc:460] Webserver started at http://127.24.162.190:34145/ using document root <none> and password file <none>
I20260808 10:03:04.418020 29596 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:04.418088 29596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:04.418228 29596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:04.419358 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "8f9039ab624141e089ec054d922436e6"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "28fed4439c387cb095e1ad363052cf5b"
server_key_iv: "1e40dc10bba29fbf42ede8802eb44f23"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.419773 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "8f9039ab624141e089ec054d922436e6"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "28fed4439c387cb095e1ad363052cf5b"
server_key_iv: "1e40dc10bba29fbf42ede8802eb44f23"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.421167 29596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:03:04.421912 29614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.422076 29596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:04.422178 29596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "8f9039ab624141e089ec054d922436e6"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "28fed4439c387cb095e1ad363052cf5b"
server_key_iv: "1e40dc10bba29fbf42ede8802eb44f23"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.422266 29596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:04.430223 29596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:04.432257 29596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:04.432422 29596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:04.436694 29596 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:39943
I20260808 10:03:04.436758 29684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:39943 every 8 connection(s)
I20260808 10:03:04.437098 29596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:04.438140 29686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:04.440366 29686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6: Bootstrap starting.
I20260808 10:03:04.440922 29686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:04.441169 29686 log.cc:824] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:04.441849 29686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6: No bootstrap required, opened a new log
I20260808 10:03:04.443471 29686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } }
I20260808 10:03:04.443602 29686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:04.443625 29686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9039ab624141e089ec054d922436e6, State: Initialized, Role: FOLLOWER
I20260808 10:03:04.443771 29686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } }
I20260808 10:03:04.443920 29686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:04.443946 29686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:04.443975 29686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:04.444703 29686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } }
I20260808 10:03:04.444788 29686 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f9039ab624141e089ec054d922436e6; no voters:
I20260808 10:03:04.444906 29686 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:04.444955 29691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:04.445130 29691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9039ab624141e089ec054d922436e6, State: Running, Role: LEADER
I20260808 10:03:04.445205 29686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:04.445288 29691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } }
I20260808 10:03:04.445681 29692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f9039ab624141e089ec054d922436e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } } }
I20260808 10:03:04.445681 29693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9039ab624141e089ec054d922436e6. Latest consensus state: current_term: 1 leader_uuid: "8f9039ab624141e089ec054d922436e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9039ab624141e089ec054d922436e6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 39943 } } }
I20260808 10:03:04.445873 29693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:04.446123 29692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:04.446393 29705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:04.446664 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29596
I20260808 10:03:04.446776 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:04.446962 29705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:04.447062 25226 external_mini_cluster.cc:1428] Setting key 02d4fe69b612569abfcb871c1a78e571
I20260808 10:03:04.449095 29705 catalog_manager.cc:1347] Generated new cluster ID: de3ba3712c984d87a6eeadaa064905a1
I20260808 10:03:04.449146 29705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:04.460960 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.447523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.24.162.190:39943)
0808 10:03:04.447725 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:04.447734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:04.447821 (+ 87us) server_negotiation.cc:407] Connection header received
0808 10:03:04.447973 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:04.447980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:04.448032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:04.448171 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:04.448787 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.450182 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.451229 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.451906 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.452026 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.454864 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:04.454869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:04.454869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:04.454881 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:04.459256 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.459696 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.459699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.459700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.459801 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.460029 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.460030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.460031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.460314 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:04.460454 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:04.460577 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:03:04.460719 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260808 10:03:04.472028 29705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:04.472596 29705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:04.477368 29705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f9039ab624141e089ec054d922436e6: Generated new TSK 0
I20260808 10:03:04.477573 29705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:04.507709 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:04.590801 29722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:04.591076 29722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:04.591104 29722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:04.592304 29722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:04.592351 29722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:04.592378 29722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:04.593840 29722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:04.594086 29722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:04.594333 29722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:04.599844 29722 server_base.cc:1034] running on GCE node
W20260808 10:03:04.599836 29732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.599836 29731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.599834 29734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:04.600329 29722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:04.600596 29722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:04.601796 29722 hybrid_clock.cc:648] HybridClock initialized: now 1786183384601785 us; error 26 us; skew 500 ppm
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:04.604514 29722 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:04.605064 29722 webserver.cc:460] Webserver started at http://127.24.162.129:40591/ using document root <none> and password file <none>
I20260808 10:03:04.605315 29722 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:04.605427 29722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:04.605602 29722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:04.607149 29722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "57c315116af44dbb8c5a278d85341c70"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "25aa684e1779b8be303386aa09c2d15a"
server_key_iv: "f3c9511c4310f282966ae0886f819f66"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.607560 29722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "57c315116af44dbb8c5a278d85341c70"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "25aa684e1779b8be303386aa09c2d15a"
server_key_iv: "f3c9511c4310f282966ae0886f819f66"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.608896 29722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:04.609620 29744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.609798 29722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:04.609896 29722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "57c315116af44dbb8c5a278d85341c70"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "25aa684e1779b8be303386aa09c2d15a"
server_key_iv: "f3c9511c4310f282966ae0886f819f66"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.609979 29722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:04.618250 29722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:04.620483 29722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:04.620647 29722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:04.620869 29722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:04.621155 29722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:04.621205 29722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.621239 29722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:04.621265 29722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.627457 29722 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:44501
I20260808 10:03:04.627518 29913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:44501 every 8 connection(s)
I20260808 10:03:04.627859 29722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:04.632910 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29722
I20260808 10:03:04.633041 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:04.633265 25226 external_mini_cluster.cc:1428] Setting key 0f8042643d5392941a19ac8023e8fb70
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:04.637432 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.628561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:51935 (local address 127.24.162.190:39943)
0808 10:03:04.628603 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:04.628605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:04.628791 (+ 186us) server_negotiation.cc:407] Connection header received
0808 10:03:04.629256 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:04.629258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:04.629272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:04.629325 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:04.630182 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.630689 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.631697 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.631907 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.631963 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.634244 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:04.634245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:04.634245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:04.634251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:04.635983 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.636414 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.636415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.636415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.636484 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.636719 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.636720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.636720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.636886 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:04.636963 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:04.637294 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 10:03:04.637369 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:03:04.637781 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.628529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:39943 (local address 127.24.162.129:51935)
0808 10:03:04.628676 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:04.628709 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:04.629151 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:04.629375 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:04.629379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:04.629930 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:04.630103 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.630107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.630769 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.630770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.631624 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.631626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.631984 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.631985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.632080 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.632484 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:04.632492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:04.634133 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:04.636073 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.636081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.636082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.636335 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.636553 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.636554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.636554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.636651 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.636968 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:04.636975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:04.637145 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 10:03:04.637272 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 10:03:04.637451 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260808 10:03:04.638357 29914 heartbeater.cc:347] Connected to a master server at 127.24.162.190:39943
I20260808 10:03:04.638477 29914 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:04.638654 29914 heartbeater.cc:511] Master 127.24.162.190:39943 requested a full tablet report, sending...
I20260808 10:03:04.639211 29636 ts_manager.cc:194] Registered new tserver with Master: 57c315116af44dbb8c5a278d85341c70 (127.24.162.129:44501)
I20260808 10:03:04.639832 29636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:51935
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:04.661195 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:04.743023 29920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:04.743237 29920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:04.743261 29920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:04.744452 29920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:04.744508 29920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:04.744554 29920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:04.746335 29920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:04.746527 29920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:04.746712 29920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:04.751753 29934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.751777 29930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.751757 29932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:04.751961 29920 server_base.cc:1034] running on GCE node
I20260808 10:03:04.752251 29920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:04.752521 29920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:04.753684 29920 hybrid_clock.cc:648] HybridClock initialized: now 1786183384753679 us; error 39 us; skew 500 ppm
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:04.756312 29920 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:04.756754 29920 webserver.cc:460] Webserver started at http://127.24.162.130:33241/ using document root <none> and password file <none>
I20260808 10:03:04.757021 29920 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:04.757091 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:04.757251 29920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:04.758306 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "9eff6a3ac7114ec985ec1f1d5f623619"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "ef7cdef12ccdfa89b23299afab9d2c8c"
server_key_iv: "f1f32e27ab23920870e528b200324330"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.758689 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "9eff6a3ac7114ec985ec1f1d5f623619"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "ef7cdef12ccdfa89b23299afab9d2c8c"
server_key_iv: "f1f32e27ab23920870e528b200324330"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.760074 29920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:03:04.760815 29945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.760983 29920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.761085 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "9eff6a3ac7114ec985ec1f1d5f623619"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "ef7cdef12ccdfa89b23299afab9d2c8c"
server_key_iv: "f1f32e27ab23920870e528b200324330"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.761166 29920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:04.771720 29920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:04.774024 29920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:04.774187 29920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:04.774410 29920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:04.774720 29920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:04.774777 29920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.774818 29920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:04.774848 29920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.780862 29920 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:37503
I20260808 10:03:04.780923 30112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:37503 every 8 connection(s)
I20260808 10:03:04.781251 29920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:04.786092 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 29920
I20260808 10:03:04.786175 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:04.786381 25226 external_mini_cluster.cc:1428] Setting key c556f4db06e7d0a39818b38581b706a6
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:04.791417 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.782074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:60579 (local address 127.24.162.190:39943)
0808 10:03:04.782120 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:04.782122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:04.782347 (+ 225us) server_negotiation.cc:407] Connection header received
0808 10:03:04.782875 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:04.782877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:04.782891 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:04.782986 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:04.783935 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.784441 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.785444 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.785655 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.785746 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.788087 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:04.788088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:04.788089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:04.788097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:04.789866 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.790260 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.790262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.790262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.790336 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.790581 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.790582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.790582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.790734 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:04.790806 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:04.791147 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 10:03:04.791299 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:04.791510 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.782031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:39943 (local address 127.24.162.130:60579)
0808 10:03:04.782216 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:04.782251 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:04.782759 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:04.783057 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:04.783061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:04.783639 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:04.783825 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.783830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.784537 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.784539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.785347 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.785350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.785755 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.785756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.785892 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.786444 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:04.786452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:04.787991 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:04.789955 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.789961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.789963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.790184 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.790405 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.790406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.790406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.790512 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.790799 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:04.790803 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:04.790951 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 10:03:04.791099 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 10:03:04.791282 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260808 10:03:04.792357 30113 heartbeater.cc:347] Connected to a master server at 127.24.162.190:39943
I20260808 10:03:04.792454 30113 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:04.792642 30113 heartbeater.cc:511] Master 127.24.162.190:39943 requested a full tablet report, sending...
I20260808 10:03:04.793118 29636 ts_manager.cc:194] Registered new tserver with Master: 9eff6a3ac7114ec985ec1f1d5f623619 (127.24.162.130:37503)
I20260808 10:03:04.793646 29636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:60579
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:04.815771 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:04.899370 30120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:04.899562 30120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:04.899585 30120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:04.900722 30120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:04.900763 30120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:04.900789 30120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:04.902226 30120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:39943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:04.902458 30120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:04.902705 30120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:04.907603 30130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:04.907635 30126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:04.907696 30120 server_base.cc:1034] running on GCE node
W20260808 10:03:04.907601 30127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:04.908071 30120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:04.908317 30120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:04.909482 30120 hybrid_clock.cc:648] HybridClock initialized: now 1786183384909467 us; error 27 us; skew 500 ppm
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:04.912072 30120 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:04.912482 30120 webserver.cc:460] Webserver started at http://127.24.162.131:39381/ using document root <none> and password file <none>
I20260808 10:03:04.912703 30120 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:04.912770 30120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:04.912930 30120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:04.913940 30120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "2141ff4f7c2a471fa2b7e893fd0f6ccf"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "c5d48e905c946a55c3c7d8b5749a61d0"
server_key_iv: "7e8c3a00ac63fb067dbb40d130276712"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.914299 30120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "2141ff4f7c2a471fa2b7e893fd0f6ccf"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "c5d48e905c946a55c3c7d8b5749a61d0"
server_key_iv: "7e8c3a00ac63fb067dbb40d130276712"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.915699 30120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:04.916404 30141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.916574 30120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.916680 30120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "2141ff4f7c2a471fa2b7e893fd0f6ccf"
format_stamp: "Formatted at 2026-08-08 10:03:04 on dist-test-slave-nfb5"
server_key: "c5d48e905c946a55c3c7d8b5749a61d0"
server_key_iv: "7e8c3a00ac63fb067dbb40d130276712"
server_key_version: "encryptionkey@0"
I20260808 10:03:04.916754 30120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:04.939600 30120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:04.941525 30120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:04.941705 30120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:04.941928 30120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:04.942232 30120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:04.942287 30120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.942330 30120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:04.942378 30120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:04.948622 30120 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33481
I20260808 10:03:04.948700 30293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33481 every 8 connection(s)
I20260808 10:03:04.949025 30120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:04.951126 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 30120
I20260808 10:03:04.951207 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:04.951404 25226 external_mini_cluster.cc:1428] Setting key effea4ba76be407fe9edf29f5eb04bfa
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:04.958218 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.949757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:48129 (local address 127.24.162.190:39943)
0808 10:03:04.949801 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:04.949804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:04.949969 (+ 165us) server_negotiation.cc:407] Connection header received
0808 10:03:04.950466 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:04.950467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:04.950476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:04.950534 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:04.951607 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.952384 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.953306 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.953485 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.953539 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.955564 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:04.955565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:04.955566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:04.955572 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:04.957080 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.957418 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.957419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.957419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.957472 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.957672 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.957672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.957673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.957813 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:04.957865 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:04.958092 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 10:03:04.958152 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:04.958386 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.949699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:39943 (local address 127.24.162.131:48129)
0808 10:03:04.949859 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:04.949900 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:04.950381 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:04.950584 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:04.950587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:04.951274 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:04.951523 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.951533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.952467 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.952468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.953230 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:04.953232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.953559 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.953560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:04.953659 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.954090 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:04.954098 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:04.955481 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:04.957148 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.957153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.957154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.957356 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.957528 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:04.957529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:04.957529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:04.957619 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:04.957870 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:04.957874 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:04.957978 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 10:03:04.958058 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:03:04.958167 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260808 10:03:04.958894 30294 heartbeater.cc:347] Connected to a master server at 127.24.162.190:39943
I20260808 10:03:04.959007 30294 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:04.959144 30294 heartbeater.cc:511] Master 127.24.162.190:39943 requested a full tablet report, sending...
I20260808 10:03:04.959508 29636 ts_manager.cc:194] Registered new tserver with Master: 2141ff4f7c2a471fa2b7e893fd0f6ccf (127.24.162.131:33481)
I20260808 10:03:04.959877 29636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:48129
I20260808 10:03:04.963138 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 10:03:04 dist-test-slave-nfb5 krb5kdc[29577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183384, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:04.993549 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:04.986099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.24.162.190:39943)
0808 10:03:04.986160 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:04.986162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:04.986233 (+ 71us) server_negotiation.cc:407] Connection header received
0808 10:03:04.986283 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:04.986284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:04.986293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:04.986343 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:04.986902 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.987493 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.988224 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:04.988408 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:04.988472 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:04.990793 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:04.990795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:04.990795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:04.990803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:04.992313 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.992705 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.992706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.992707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.992770 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:04.992977 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:04.992978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:04.992979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:04.993145 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:04.993256 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:04.993357 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:03:04.993441 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 10:03:04.993816 29636 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59632
I20260808 10:03:04.994282 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29722
I20260808 10:03:05.009510 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29920
I20260808 10:03:05.014878 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 30120
I20260808 10:03:05.022590 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 29596
2026-08-08T10:03:05Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2050 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:05 dist-test-slave-nfb5 krb5kdc[30309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:05 dist-test-slave-nfb5 krb5kdc[30309](info): set up 2 sockets
Aug 08 10:03:05 dist-test-slave-nfb5 krb5kdc[30309](info): commencing operation
krb5kdc: starting...
W20260808 10:03:06.284587 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:06.405896 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44603
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:44603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:06.488535 30330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:06.488801 30330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:06.488839 30330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:06.490010 30330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:06.490082 30330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:06.490104 30330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:06.490125 30330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:06.490157 30330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:06.491740 30330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:44603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:06.492025 30330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:06.492255 30330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:06.497154 30338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:06.497094 30336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:06.497920 30335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:06.498054 30330 server_base.cc:1034] running on GCE node
I20260808 10:03:06.498306 30330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:06.498596 30330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:06.499783 30330 hybrid_clock.cc:648] HybridClock initialized: now 1786183386499764 us; error 49 us; skew 500 ppm
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:06.502230 30330 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:06.502719 30330 webserver.cc:460] Webserver started at http://127.24.162.190:34113/ using document root <none> and password file <none>
I20260808 10:03:06.503014 30330 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:06.503083 30330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:06.503332 30330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:06.504429 30330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e58a828928a0c67f23bf4f1895ecd650"
server_key_iv: "1b4178ea05346fc576781a096f346504"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.504822 30330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e58a828928a0c67f23bf4f1895ecd650"
server_key_iv: "1b4178ea05346fc576781a096f346504"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.506212 30330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:06.506975 30349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.507150 30330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:06.507256 30330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e58a828928a0c67f23bf4f1895ecd650"
server_key_iv: "1b4178ea05346fc576781a096f346504"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.507339 30330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:06.526834 30330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:06.528874 30330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:06.529047 30330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:06.533352 30330 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:44603
I20260808 10:03:06.533397 30428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:44603 every 8 connection(s)
I20260808 10:03:06.533968 30330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:06.534654 30430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:06.537092 30430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: Bootstrap starting.
I20260808 10:03:06.537709 30430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:06.538056 30430 log.cc:824] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:06.538777 30430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: No bootstrap required, opened a new log
I20260808 10:03:06.540383 30430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } }
I20260808 10:03:06.540521 30430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:06.540544 30430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263e1ccbf5ed4366a27b8740e2ddf6ef, State: Initialized, Role: FOLLOWER
I20260808 10:03:06.540707 30430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } }
I20260808 10:03:06.540809 30430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:06.540864 30430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:06.540917 30430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:06.541033 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 30330
I20260808 10:03:06.541154 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:06.541343 25226 external_mini_cluster.cc:1428] Setting key cfa0a8a3028aec5509956532bfc6fc7a
I20260808 10:03:06.542142 30430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } }
I20260808 10:03:06.542235 30430 leader_election.cc:302] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 263e1ccbf5ed4366a27b8740e2ddf6ef; no voters:
I20260808 10:03:06.542418 30430 leader_election.cc:288] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:06.542454 30436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:06.542569 30436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [term 1 LEADER]: Becoming Leader. State: Replica: 263e1ccbf5ed4366a27b8740e2ddf6ef, State: Running, Role: LEADER
I20260808 10:03:06.542683 30430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:06.542661 30436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } }
I20260808 10:03:06.543037 30437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 263e1ccbf5ed4366a27b8740e2ddf6ef. Latest consensus state: current_term: 1 leader_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } } }
I20260808 10:03:06.543143 30437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:06.543038 30438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "263e1ccbf5ed4366a27b8740e2ddf6ef" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44603 } } }
I20260808 10:03:06.543340 30438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef [sys.catalog]: This master's current role is: LEADER
W20260808 10:03:06.545208 30459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:06.545267 30459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:06.545315 30443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:06.545454 30443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:06.547232 30443 catalog_manager.cc:1347] Generated new cluster ID: a8efe7e8d7a641fb9c747951f5ba7c88
I20260808 10:03:06.547289 30443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:06.552302 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:06.541879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.24.162.190:44603)
0808 10:03:06.542076 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:06.542084 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:06.542109 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:03:06.542233 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:06.542240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:06.542301 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:06.542480 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:06.543508 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.544501 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.545525 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.545950 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.546035 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:06.548890 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:06.548896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:06.548897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:06.548912 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:06.550648 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.551110 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.551112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.551113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.551229 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.551463 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.551464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.551464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.551698 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:06.551861 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:06.551977 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 10:03:06.552120 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260808 10:03:06.560452 30443 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:06.560986 30443 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:06.566051 30443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 263e1ccbf5ed4366a27b8740e2ddf6ef: Generated new TSK 0
I20260808 10:03:06.566234 30443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:06.596092 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:06.682447 30466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:06.682648 30466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:06.682672 30466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:06.683836 30466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:06.683878 30466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:06.683904 30466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:06.685425 30466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:06.685679 30466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:06.685925 30466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:06.690989 30476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:06.690966 30479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:06.690958 30477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:06.691267 30466 server_base.cc:1034] running on GCE node
I20260808 10:03:06.691478 30466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:06.691717 30466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:06.692880 30466 hybrid_clock.cc:648] HybridClock initialized: now 1786183386692870 us; error 32 us; skew 500 ppm
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:06.695389 30466 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:06.695859 30466 webserver.cc:460] Webserver started at http://127.24.162.129:42387/ using document root <none> and password file <none>
I20260808 10:03:06.696102 30466 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:06.696173 30466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:06.696311 30466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:06.697341 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "614e2026098f434caa695e161d0a80c0"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "bad9a89a45dce3a6a98d8002af750a5d"
server_key_iv: "0ca0a25f8589d3b9bc7855f2a87961f7"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.697706 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "614e2026098f434caa695e161d0a80c0"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "bad9a89a45dce3a6a98d8002af750a5d"
server_key_iv: "0ca0a25f8589d3b9bc7855f2a87961f7"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.699054 30466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:03:06.699818 30494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.699986 30466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:06.700088 30466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "614e2026098f434caa695e161d0a80c0"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "bad9a89a45dce3a6a98d8002af750a5d"
server_key_iv: "0ca0a25f8589d3b9bc7855f2a87961f7"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.700166 30466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:06.713377 30466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:06.715354 30466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:06.715520 30466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:06.715754 30466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:06.716061 30466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:06.716118 30466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.716154 30466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:06.716183 30466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.722041 30466 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:34081
I20260808 10:03:06.722115 30677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:34081 every 8 connection(s)
I20260808 10:03:06.722445 30466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:06.731472 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 30466
I20260808 10:03:06.731565 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:06.731757 25226 external_mini_cluster.cc:1428] Setting key 90f382b06ff6c98c83a7aa28855f2077
I20260808 10:03:06.731926 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:06.723283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:34599 (local address 127.24.162.190:44603)
0808 10:03:06.723332 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:06.723334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:06.723532 (+ 198us) server_negotiation.cc:407] Connection header received
0808 10:03:06.724490 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:06.724491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:06.724503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:06.724565 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:06.725165 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.725668 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.726631 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.726839 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.726892 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:06.729159 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:06.729161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:06.729161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:06.729168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:06.730407 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.730825 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.730826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.730826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.730879 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.731165 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.731165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.731166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.731331 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:06.731408 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:06.731733 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 10:03:06.731833 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:03:06.732131 30682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:06.723231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44603 (local address 127.24.162.129:34599)
0808 10:03:06.723409 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:06.723453 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:06.724386 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:06.724642 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:06.724645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:06.724903 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:06.725090 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:06.725093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.725751 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.725752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:06.726554 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:06.726557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.726958 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.726960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:06.727101 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:06.727590 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:06.727598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:06.729060 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:06.730491 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:06.730498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:06.730501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:06.730751 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:06.731000 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:06.731001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:06.731001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:06.731108 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:06.731403 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:06.731408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:06.731545 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 10:03:06.731700 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 10:03:06.731877 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260808 10:03:06.732668 30680 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44603
I20260808 10:03:06.732769 30680 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:06.732970 30680 heartbeater.cc:511] Master 127.24.162.190:44603 requested a full tablet report, sending...
I20260808 10:03:06.733500 30369 ts_manager.cc:194] Registered new tserver with Master: 614e2026098f434caa695e161d0a80c0 (127.24.162.129:34081)
I20260808 10:03:06.734154 30369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:34599
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:06.757737 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:06.842173 30686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:06.842406 30686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:06.842444 30686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:06.843648 30686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:06.843710 30686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:06.843755 30686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:06.845386 30686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:06.845652 30686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:06.845883 30686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:06.850713 30698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:06.850747 30694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:06.850986 30686 server_base.cc:1034] running on GCE node
W20260808 10:03:06.850839 30700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:06.851311 30686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:06.851560 30686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:06.852718 30686 hybrid_clock.cc:648] HybridClock initialized: now 1786183386852717 us; error 36 us; skew 500 ppm
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:06.855208 30686 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:06.855645 30686 webserver.cc:460] Webserver started at http://127.24.162.130:37833/ using document root <none> and password file <none>
I20260808 10:03:06.855881 30686 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:06.855950 30686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:06.856088 30686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:06.857092 30686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "eaa4165dc1fc4864a4e44d99df597ee4"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e71183e069c0569b4080568045b04513"
server_key_iv: "81c77e6ab7a8cf025fc3b78dd2f96db0"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.857450 30686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "eaa4165dc1fc4864a4e44d99df597ee4"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e71183e069c0569b4080568045b04513"
server_key_iv: "81c77e6ab7a8cf025fc3b78dd2f96db0"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.858762 30686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:03:06.859539 30716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.859715 30686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.859812 30686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "eaa4165dc1fc4864a4e44d99df597ee4"
format_stamp: "Formatted at 2026-08-08 10:03:06 on dist-test-slave-nfb5"
server_key: "e71183e069c0569b4080568045b04513"
server_key_iv: "81c77e6ab7a8cf025fc3b78dd2f96db0"
server_key_version: "encryptionkey@0"
I20260808 10:03:06.859895 30686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:06.877771 30686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:06.879671 30686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:06.879833 30686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:06.880072 30686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:06.880378 30686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:06.880430 30686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.880472 30686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:06.880498 30686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:06.886284 30686 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:39097
I20260808 10:03:06.886323 30912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:39097 every 8 connection(s)
I20260808 10:03:06.886652 30686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:03:06 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:06.892833 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 30686
I20260808 10:03:06.892948 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:06.893146 25226 external_mini_cluster.cc:1428] Setting key cd3ba9ca43ea7cb16aaa7caa6f9a6f39
I20260808 10:03:06.896621 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:06.887452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:52857 (local address 127.24.162.190:44603)
0808 10:03:06.887528 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:06.887531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:06.887742 (+ 211us) server_negotiation.cc:407] Connection header received
0808 10:03:06.888631 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:06.888632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:06.888643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:06.888693 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:06.889279 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.889775 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.890713 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.890909 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.891024 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:06.893314 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:06.893315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:06.893316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:06.893323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:06.895134 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.895537 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.895538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.895538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.895605 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:06.895927 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:06.895928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:06.895928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:06.896041 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:06.896166 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:06.896482 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 10:03:06.896554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:03:06.896912 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:06.887416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44603 (local address 127.24.162.130:52857)
0808 10:03:06.887620 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:06.887656 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:06.888545 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:06.888772 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:06.888775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:06.889031 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:06.889210 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:06.889213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.889851 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.889852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:06.890632 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:06.890634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:06.891033 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:06.891034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:06.891140 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:06.891676 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:06.891690 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:06.893204 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:06.895236 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:06.895242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:06.895244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:06.895461 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:06.895698 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:06.895699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:06.895699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:06.895854 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:06.896157 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:06.896163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:06.896331 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 10:03:06.896454 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:03:06.896633 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260808 10:03:06.897394 30913 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44603
I20260808 10:03:06.897472 30913 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:06.897665 30913 heartbeater.cc:511] Master 127.24.162.190:44603 requested a full tablet report, sending...
I20260808 10:03:06.898092 30369 ts_manager.cc:194] Registered new tserver with Master: eaa4165dc1fc4864a4e44d99df597ee4 (127.24.162.130:39097)
I20260808 10:03:06.898636 30369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:52857
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:06.921303 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:07.002247 30919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:07.002480 30919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:07.002521 30919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:07.003855 30919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:07.003922 30919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:07.003968 30919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:07.005491 30919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:07.005752 30919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:07.006014 30919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:07.011111 30931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:07.011241 30928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:07.011140 30929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:07.011175 30919 server_base.cc:1034] running on GCE node
I20260808 10:03:07.011544 30919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:07.011796 30919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:07.012961 30919 hybrid_clock.cc:648] HybridClock initialized: now 1786183387012962 us; error 38 us; skew 500 ppm
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:07.015601 30919 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:07.016058 30919 webserver.cc:460] Webserver started at http://127.24.162.131:44771/ using document root <none> and password file <none>
I20260808 10:03:07.016279 30919 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:07.016323 30919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:07.016487 30919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:07.017490 30919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "6a5d3b7dbee445af9712bff082a47c21"
format_stamp: "Formatted at 2026-08-08 10:03:07 on dist-test-slave-nfb5"
server_key: "387aa2a20bd151ea33e79f4c05a0fda2"
server_key_iv: "e661f54b78fc414f0e7c0602a8808cbd"
server_key_version: "encryptionkey@0"
I20260808 10:03:07.017849 30919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "6a5d3b7dbee445af9712bff082a47c21"
format_stamp: "Formatted at 2026-08-08 10:03:07 on dist-test-slave-nfb5"
server_key: "387aa2a20bd151ea33e79f4c05a0fda2"
server_key_iv: "e661f54b78fc414f0e7c0602a8808cbd"
server_key_version: "encryptionkey@0"
I20260808 10:03:07.019200 30919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:03:07.019953 30940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:07.020139 30919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:07.020236 30919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "6a5d3b7dbee445af9712bff082a47c21"
format_stamp: "Formatted at 2026-08-08 10:03:07 on dist-test-slave-nfb5"
server_key: "387aa2a20bd151ea33e79f4c05a0fda2"
server_key_iv: "e661f54b78fc414f0e7c0602a8808cbd"
server_key_version: "encryptionkey@0"
I20260808 10:03:07.020304 30919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:07.027415 30919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:07.029666 30919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:07.029785 30919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:07.030040 30919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:07.030370 30919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:07.030404 30919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:07.030424 30919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:07.030473 30919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:07.036244 30919 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:36081
I20260808 10:03:07.036315 31107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:36081 every 8 connection(s)
I20260808 10:03:07.036631 30919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:07.046084 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:07.037405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:43645 (local address 127.24.162.190:44603)
0808 10:03:07.037483 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:07.037486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:07.037660 (+ 174us) server_negotiation.cc:407] Connection header received
0808 10:03:07.038543 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:07.038544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:07.038554 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:07.038599 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:07.039300 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.039791 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.040772 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.040956 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.041005 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:07.043110 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:07.043112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:07.043112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:07.043122 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:07.044653 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.045079 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.045080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.045081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.045158 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.045419 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.045420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.045420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.045538 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:07.045615 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:07.045885 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 10:03:07.045993 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:03:07.046173 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 30919
I20260808 10:03:07.046244 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:07.037359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44603 (local address 127.24.162.131:43645)
0808 10:03:07.037545 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:07.037580 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:07.038455 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:07.038689 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:07.038693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:07.039035 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:07.039227 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:07.039232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.039884 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.039885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:07.040695 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:07.040697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.041027 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.041028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:07.041124 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:07.041551 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:07.041559 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:07.043011 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:07.044747 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:07.044753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:07.044755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:07.044997 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:07.045228 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:07.045229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:07.045229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:07.045347 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:07.045607 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:07.045613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:07.045748 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 10:03:07.045861 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:03:07.046017 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260808 10:03:07.046296 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:07.046489 25226 external_mini_cluster.cc:1428] Setting key 1250888821fb7bc019cdb5662f8ad788
I20260808 10:03:07.046671 31112 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44603
I20260808 10:03:07.046762 31112 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:07.046967 31112 heartbeater.cc:511] Master 127.24.162.190:44603 requested a full tablet report, sending...
I20260808 10:03:07.047410 30368 ts_manager.cc:194] Registered new tserver with Master: 6a5d3b7dbee445af9712bff082a47c21 (127.24.162.131:36081)
I20260808 10:03:07.047803 30368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:43645
I20260808 10:03:07.058419 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[30309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:03:07.082192 31120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:07.074430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.24.162.129:34081)
0808 10:03:07.074599 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:07.074605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:07.074616 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:07.074642 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:07.074644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:07.074659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:07.074732 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:07.075389 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.076095 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.076829 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.077129 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.077182 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:07.079501 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:07.079510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:07.079510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:07.079517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:07.081178 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.081489 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.081492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.081492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.081560 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.081752 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.081752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.081752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.081880 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:07.081981 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:07.082051 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:03:07.082112 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[30309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:07.090140 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:07.083630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57446 (local address 127.24.162.190:44603)
0808 10:03:07.083714 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:07.083716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:07.083725 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:07.083769 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:07.083769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:07.083779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:07.083824 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:07.084267 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.084758 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.085402 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:07.085596 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:07.085645 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:07.087744 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:07.087745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:07.087746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:07.087752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:07.089165 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.089462 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.089463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.089463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.089512 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:07.089656 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:07.089657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:07.089657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:07.089810 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:07.089883 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:07.089968 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:03:07.090026 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 10:03:07.090436 30368 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57446
I20260808 10:03:07.090793 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 30466
I20260808 10:03:07.098116 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 30686
I20260808 10:03:07.108572 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 30919
I20260808 10:03:07.120378 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 30330
2026-08-08T10:03:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2094 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[31126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[31126](info): set up 2 sockets
Aug 08 10:03:07 dist-test-slave-nfb5 krb5kdc[31126](info): commencing operation
krb5kdc: starting...
W20260808 10:03:08.344877 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:08.469547 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44097
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:44097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:08.553972 31143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:08.554291 31143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:08.554333 31143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:08.555531 31143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:08.555608 31143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:08.555631 31143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:08.555670 31143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:08.555696 31143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:08.557241 31143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:44097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:08.557540 31143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:08.557796 31143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:08.562608 31149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:08.562609 31151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:08.562638 31154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:08.562769 31143 server_base.cc:1034] running on GCE node
I20260808 10:03:08.563099 31143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:08.563397 31143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:08.564560 31143 hybrid_clock.cc:648] HybridClock initialized: now 1786183388564556 us; error 42 us; skew 500 ppm
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:08.567071 31143 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:08.567437 31143 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:08.567510 31143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:08.567655 31143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:08.569128 31143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "b2811fe313824d01a39859233a241620"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "1fe123d5805a2791eb8e7150cb14e89d"
server_key_iv: "b0dfb3985c92d7b53d56f0bfdc286e30"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.569509 31143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "b2811fe313824d01a39859233a241620"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "1fe123d5805a2791eb8e7150cb14e89d"
server_key_iv: "b0dfb3985c92d7b53d56f0bfdc286e30"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.570829 31143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 10:03:08.571628 31161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.571801 31143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:08.571906 31143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "b2811fe313824d01a39859233a241620"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "1fe123d5805a2791eb8e7150cb14e89d"
server_key_iv: "b0dfb3985c92d7b53d56f0bfdc286e30"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.571980 31143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:08.593353 31143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:08.595790 31143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:08.595958 31143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:08.600106 31143 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:44097
I20260808 10:03:08.600167 31238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:44097 every 8 connection(s)
I20260808 10:03:08.600467 31143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:08.601572 31239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:08.603969 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: Bootstrap starting.
I20260808 10:03:08.604607 31239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:08.604882 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 31143
I20260808 10:03:08.605003 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:08.605003 31239 log.cc:824] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:08.605193 25226 external_mini_cluster.cc:1428] Setting key 35cb09ffaa700dbbc1a45b7ae13ec2b7
I20260808 10:03:08.605837 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: No bootstrap required, opened a new log
I20260808 10:03:08.608494 31239 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } }
I20260808 10:03:08.608664 31239 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:08.608696 31239 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2811fe313824d01a39859233a241620, State: Initialized, Role: FOLLOWER
I20260808 10:03:08.608840 31239 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } }
I20260808 10:03:08.608915 31239 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:08.608945 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:08.608989 31239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:08.609666 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } }
I20260808 10:03:08.609794 31239 leader_election.cc:302] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2811fe313824d01a39859233a241620; no voters:
I20260808 10:03:08.609997 31239 leader_election.cc:288] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:08.610023 31245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:08.610159 31245 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [term 1 LEADER]: Becoming Leader. State: Replica: b2811fe313824d01a39859233a241620, State: Running, Role: LEADER
I20260808 10:03:08.610268 31239 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:08.610244 31245 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } }
I20260808 10:03:08.610608 31245 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2811fe313824d01a39859233a241620. Latest consensus state: current_term: 1 leader_uuid: "b2811fe313824d01a39859233a241620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } } }
I20260808 10:03:08.610673 31245 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:08.611006 31246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2811fe313824d01a39859233a241620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2811fe313824d01a39859233a241620" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44097 } } }
I20260808 10:03:08.611099 31246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:08.611496 31252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:08.612071 31252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:08.613708 31252 catalog_manager.cc:1347] Generated new cluster ID: 1dc822ddc2df45fdb82bd6e4448f018f
I20260808 10:03:08.613754 31252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:08.617885 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:08.605760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.24.162.190:44097)
0808 10:03:08.605961 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:08.605971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:08.606011 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:03:08.607144 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:08.607153 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:08.607216 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:08.607375 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:08.607998 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.609395 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.610427 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.611123 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.611242 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:08.614277 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:08.614281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:08.614281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:08.614293 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:08.616250 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.616668 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.616670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.616671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.616769 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.616988 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.616989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.616989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.617262 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:08.617416 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:08.617552 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 10:03:08.617678 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260808 10:03:08.645217 31252 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:08.646010 31252 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:08.652966 31252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: Generated new TSK 0
I20260808 10:03:08.653168 31252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:08.683724 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:08.766732 31275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:08.766992 31275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:08.767047 31275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:08.768277 31275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:08.768323 31275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:08.768352 31275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:08.769887 31275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:08.770134 31275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:08.770385 31275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:08.775480 31290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:08.775493 31285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:08.775490 31287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:08.775777 31275 server_base.cc:1034] running on GCE node
I20260808 10:03:08.775995 31275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:08.776264 31275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:08.777455 31275 hybrid_clock.cc:648] HybridClock initialized: now 1786183388777437 us; error 32 us; skew 500 ppm
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:08.780072 31275 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:08.780478 31275 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:08.780552 31275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:08.780730 31275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:08.781772 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "7415524f37894bc1bc2c0d1952804b6c"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "9efe13b7822cbbd2fc55a7b206ea7415"
server_key_iv: "ff7a1077041f95250748b794a2998695"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.782145 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "7415524f37894bc1bc2c0d1952804b6c"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "9efe13b7822cbbd2fc55a7b206ea7415"
server_key_iv: "ff7a1077041f95250748b794a2998695"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.783569 31275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:08.784358 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.784556 31275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:08.784691 31275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "7415524f37894bc1bc2c0d1952804b6c"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "9efe13b7822cbbd2fc55a7b206ea7415"
server_key_iv: "ff7a1077041f95250748b794a2998695"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.784768 31275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:08.803956 31275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:08.805855 31275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:08.806025 31275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:08.806262 31275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:08.806589 31275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:08.806649 31275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.806689 31275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:08.806717 31275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.813264 31275 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41143
I20260808 10:03:08.813313 31461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41143 every 8 connection(s)
I20260808 10:03:08.813591 31275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:08.818976 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 31275
I20260808 10:03:08.819103 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:08.819298 25226 external_mini_cluster.cc:1428] Setting key b4d4399da80691f8d67f8d982cc05e3f
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:08.823287 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:08.814299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:34891 (local address 127.24.162.190:44097)
0808 10:03:08.814386 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:08.814388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:08.814566 (+ 178us) server_negotiation.cc:407] Connection header received
0808 10:03:08.815116 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:08.815117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:08.815127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:08.815180 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:08.816085 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.816594 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.817589 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.817775 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.817832 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:08.820324 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:08.820326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:08.820326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:08.820333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:08.821817 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.822252 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.822254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.822254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.822316 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.822579 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.822580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.822581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.822716 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:08.822793 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:08.823140 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 10:03:08.823220 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:03:08.823566 31466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:08.814250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44097 (local address 127.24.162.129:34891)
0808 10:03:08.814436 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:08.814472 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:08.815024 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:08.815235 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:08.815239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:08.815503 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:08.816008 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:08.816012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.816678 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.816679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:08.817513 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:08.817515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.817848 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.817849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:08.817949 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:08.818504 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:08.818513 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:08.820222 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:08.821923 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:08.821930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:08.821933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:08.822157 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:08.822386 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:08.822387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:08.822387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:08.822507 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:08.822795 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:08.822803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:08.822991 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 10:03:08.823105 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:03:08.823292 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260808 10:03:08.824138 31463 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44097
I20260808 10:03:08.824234 31463 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:08.824410 31463 heartbeater.cc:511] Master 127.24.162.190:44097 requested a full tablet report, sending...
I20260808 10:03:08.824954 31180 ts_manager.cc:194] Registered new tserver with Master: 7415524f37894bc1bc2c0d1952804b6c (127.24.162.129:41143)
I20260808 10:03:08.825582 31180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:34891
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:08.847895 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:08.929773 31473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:08.929976 31473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:08.929999 31473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:08.931089 31473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:08.931123 31473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:08.931149 31473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:08.932590 31473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:08.932772 31473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:08.932952 31473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:08.937925 31482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:08.938038 31479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:08.938241 31473 server_base.cc:1034] running on GCE node
W20260808 10:03:08.937924 31478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:08.938498 31473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:08.938689 31473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:08.939868 31473 hybrid_clock.cc:648] HybridClock initialized: now 1786183388939848 us; error 31 us; skew 500 ppm
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:08.942500 31473 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:08.942965 31473 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:08.943028 31473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:08.943194 31473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:08.944232 31473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "737b0afdde694426ab19448fd64fa850"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "849d2e8b0499e5d42be0a8964fcc5829"
server_key_iv: "0491531fe40f737c03fcf59df3ad89b4"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.944643 31473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "737b0afdde694426ab19448fd64fa850"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "849d2e8b0499e5d42be0a8964fcc5829"
server_key_iv: "0491531fe40f737c03fcf59df3ad89b4"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.945947 31473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:08.946666 31489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.946843 31473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:08.946972 31473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "737b0afdde694426ab19448fd64fa850"
format_stamp: "Formatted at 2026-08-08 10:03:08 on dist-test-slave-nfb5"
server_key: "849d2e8b0499e5d42be0a8964fcc5829"
server_key_iv: "0491531fe40f737c03fcf59df3ad89b4"
server_key_version: "encryptionkey@0"
I20260808 10:03:08.947043 31473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:08.962246 31473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:08.964121 31473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:08.964231 31473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:08.964402 31473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:08.964658 31473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:08.964695 31473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.964716 31473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:08.964730 31473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:08.970275 31473 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:44303
I20260808 10:03:08.970360 31644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:44303 every 8 connection(s)
I20260808 10:03:08.970629 31473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:08.972985 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 31473
I20260808 10:03:08.973114 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:08.973299 25226 external_mini_cluster.cc:1428] Setting key aeb704a12eb3cffe01ca82bc65e67203
Aug 08 10:03:08 dist-test-slave-nfb5 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183388, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:08.980229 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:08.971399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:46903 (local address 127.24.162.190:44097)
0808 10:03:08.971462 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:08.971464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:08.971666 (+ 202us) server_negotiation.cc:407] Connection header received
0808 10:03:08.972243 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:08.972243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:08.972252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:08.972309 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:08.973221 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.973785 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.974842 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.975119 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.975197 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:08.977319 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:08.977320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:08.977320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:08.977328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:08.978843 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.979265 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.979266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.979266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.979324 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:08.979586 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:08.979587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:08.979587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:08.979717 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:08.979788 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:08.980059 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 10:03:08.980135 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:03:08.980473 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:08.971363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44097 (local address 127.24.162.130:46903)
0808 10:03:08.971538 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:08.971579 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:08.972150 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:08.972360 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:08.972363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:08.972605 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:08.973122 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:08.973129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.973881 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.973882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:08.974749 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:08.974752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:08.975210 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:08.975211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:08.975342 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:08.975898 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:08.975906 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:08.977222 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:08.978947 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:08.978954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:08.978956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:08.979180 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:08.979399 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:08.979400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:08.979400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:08.979510 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:08.979791 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:08.979798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:08.979928 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 10:03:08.980036 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:03:08.980199 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260808 10:03:08.980928 31645 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44097
I20260808 10:03:08.981020 31645 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:08.981191 31645 heartbeater.cc:511] Master 127.24.162.190:44097 requested a full tablet report, sending...
I20260808 10:03:08.981564 31180 ts_manager.cc:194] Registered new tserver with Master: 737b0afdde694426ab19448fd64fa850 (127.24.162.130:44303)
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
I20260808 10:03:08.981969 31180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:46903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:08.999882 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:09.081696 31656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:09.081924 31656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:09.081969 31656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:09.083166 31656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:09.083227 31656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:09.083271 31656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:09.084776 31656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:44097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:09.085049 31656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:09.085280 31656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:09.090212 31662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.090312 31663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.090212 31669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.090459 31656 server_base.cc:1034] running on GCE node
I20260808 10:03:09.090639 31656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:09.090823 31656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:09.091955 31656 hybrid_clock.cc:648] HybridClock initialized: now 1786183389091943 us; error 44 us; skew 500 ppm
Aug 08 10:03:09 dist-test-slave-nfb5 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183389, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:09.094372 31656 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:09.094753 31656 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:09.094828 31656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:09.095011 31656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:09.096040 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "b5eacef7e9fb4ffd887f90874471a733"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "3c282d2abe1376ce25e4a2f8295465da"
server_key_iv: "c07bd4dd9b42c762659a56c3e443b914"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.096408 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "b5eacef7e9fb4ffd887f90874471a733"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "3c282d2abe1376ce25e4a2f8295465da"
server_key_iv: "c07bd4dd9b42c762659a56c3e443b914"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.097783 31656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:09.098486 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.098659 31656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:09.098762 31656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "b5eacef7e9fb4ffd887f90874471a733"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "3c282d2abe1376ce25e4a2f8295465da"
server_key_iv: "c07bd4dd9b42c762659a56c3e443b914"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.098838 31656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:09.108655 31656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:09.110433 31656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:09.110580 31656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:09.110788 31656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:09.111207 31656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:09.111267 31656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.111306 31656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:09.111342 31656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.117458 31656 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:38679
I20260808 10:03:09.117498 31851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:38679 every 8 connection(s)
I20260808 10:03:09.117743 31656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:03:09 dist-test-slave-nfb5 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183389, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:09.124595 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 31656
I20260808 10:03:09.124715 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:09.124920 25226 external_mini_cluster.cc:1428] Setting key 1602070094395ce40fce88d2037e4ff0
I20260808 10:03:09.126765 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.118394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:57989 (local address 127.24.162.190:44097)
0808 10:03:09.118474 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.118477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.118679 (+ 202us) server_negotiation.cc:407] Connection header received
0808 10:03:09.119189 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.119191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.119201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.119253 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.120074 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.120560 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.121509 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.121685 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.121735 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.123866 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.123868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.123869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:09.123877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.125493 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:09.125986 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:09.125987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:09.125988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:09.126041 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:09.126253 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:09.126253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:09.126253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:09.126359 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.126411 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.126640 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.126701 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:03:09.126917 31857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.118370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44097 (local address 127.24.162.131:57989)
0808 10:03:09.118559 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.118593 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.119104 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.119303 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.119306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.119539 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.120001 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.120006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.120646 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.120647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.121436 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.121438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.121757 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.121758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.121853 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.122346 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:09.122355 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.123759 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.125587 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:09.125594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:09.125596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:09.125912 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:09.126092 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:09.126093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:09.126093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:09.126199 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:09.126418 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.126422 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.126532 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.126613 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.126729 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260808 10:03:09.127369 31854 heartbeater.cc:347] Connected to a master server at 127.24.162.190:44097
I20260808 10:03:09.127475 31854 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:09.127624 31854 heartbeater.cc:511] Master 127.24.162.190:44097 requested a full tablet report, sending...
I20260808 10:03:09.127977 31180 ts_manager.cc:194] Registered new tserver with Master: b5eacef7e9fb4ffd887f90874471a733 (127.24.162.131:38679)
I20260808 10:03:09.128322 31180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:57989
I20260808 10:03:09.136754 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:03:09.143970 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.138059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.24.162.190:44097)
0808 10:03:09.138123 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.138125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.138182 (+ 57us) server_negotiation.cc:407] Connection header received
0808 10:03:09.138238 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.138239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.138247 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.138285 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.138826 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.139350 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.140041 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.140213 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.140260 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.141555 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.141556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.141556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:09.141562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.142741 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:09.143208 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:09.143209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:09.143209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:09.143263 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:09.143440 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:09.143441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:09.143441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:09.143577 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.143654 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.143744 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.143803 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:03:09.145653 31180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:09.147272 31180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:09.153899 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.24.162.131:38679)
0808 10:03:09.149985 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.149990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.150461 (+ 471us) server_negotiation.cc:407] Connection header received
0808 10:03:09.150505 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.150508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.150524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.150589 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:09.150785 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151559 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.152829 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.153487 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153560 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.153611 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.153767 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.153804 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260808 10:03:09.153903 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:38679 (local address 127.0.0.1:49524)
0808 10:03:09.150313 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.150323 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.150392 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.150598 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.150600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.150602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:09.150694 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.150697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151662 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.151663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.152742 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.152745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.153558 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.153639 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.153653 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.153732 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.153796 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":26,"thread_start_us":68,"threads_started":1}
I20260808 10:03:09.154126 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:44303 (local address 127.0.0.1:45574)
0808 10:03:09.149801 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.149835 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.149902 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.150339 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.150340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.150342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:09.150380 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.150382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151704 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.151705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.153014 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.153017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.153903 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.153966 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.153970 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.154028 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.154068 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":95,"thread_start_us":59,"threads_started":1}
I20260808 10:03:09.154177 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.24.162.130:44303)
0808 10:03:09.150048 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.150053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.150064 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:09.150105 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.150107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.150123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.150206 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:09.150466 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151586 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153121 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.153759 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153820 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.153869 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.154044 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.154080 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":111,"threads_started":1}
I20260808 10:03:09.154721 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41143 (local address 127.0.0.1:48626)
0808 10:03:09.149787 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.149829 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.149905 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.150232 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.150235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.150238 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:09.150307 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.150309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151704 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.151705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.153015 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.153017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.154480 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.154481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.154561 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.154564 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.154612 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.154651 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":99,"thread_start_us":67,"threads_started":1}
I20260808 10:03:09.154693 31755 tablet_service.cc:1467] Processing CreateTablet for tablet dd2caf2b1e164bed8c184ad788dd469d (DEFAULT_TABLE table=test-table [id=183cd34f62ba4e04abdc0d03a132b667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.154906 31547 tablet_service.cc:1467] Processing CreateTablet for tablet dd2caf2b1e164bed8c184ad788dd469d (DEFAULT_TABLE table=test-table [id=183cd34f62ba4e04abdc0d03a132b667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.155041 31755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2caf2b1e164bed8c184ad788dd469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.155189 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.149825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.24.162.129:41143)
0808 10:03:09.150053 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.150056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.150065 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:09.150105 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.150107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.150123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.150207 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:09.150403 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.151581 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153104 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.153836 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.153922 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.155023 (+ 1101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.155054 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.155104 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260808 10:03:09.155303 31547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2caf2b1e164bed8c184ad788dd469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.155764 31366 tablet_service.cc:1467] Processing CreateTablet for tablet dd2caf2b1e164bed8c184ad788dd469d (DEFAULT_TABLE table=test-table [id=183cd34f62ba4e04abdc0d03a132b667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.156114 31366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd2caf2b1e164bed8c184ad788dd469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.157878 31873 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Bootstrap starting.
I20260808 10:03:09.158584 31874 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Bootstrap starting.
I20260808 10:03:09.158685 31873 tablet_bootstrap.cc:654] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.159250 31874 tablet_bootstrap.cc:654] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.159300 31873 log.cc:824] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.159659 31875 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Bootstrap starting.
I20260808 10:03:09.159885 31874 log.cc:824] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.160219 31875 tablet_bootstrap.cc:654] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.160610 31874 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: No bootstrap required, opened a new log
I20260808 10:03:09.160679 31874 ts_tablet_manager.cc:1397] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:09.160784 31875 log.cc:824] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.161628 31875 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: No bootstrap required, opened a new log
I20260808 10:03:09.161712 31875 ts_tablet_manager.cc:1397] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:09.162355 31874 raft_consensus.cc:348] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.162483 31874 raft_consensus.cc:374] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.162505 31874 raft_consensus.cc:729] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5eacef7e9fb4ffd887f90874471a733, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.162629 31874 consensus_queue.cc:260] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.162963 31874 ts_tablet_manager.cc:1428] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:03:09.163002 31854 heartbeater.cc:503] Master 127.24.162.190:44097 was elected leader, sending a full tablet report...
I20260808 10:03:09.163393 31875 raft_consensus.cc:348] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.163537 31875 raft_consensus.cc:374] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.163582 31875 raft_consensus.cc:729] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7415524f37894bc1bc2c0d1952804b6c, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.163686 31875 consensus_queue.cc:260] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.163973 31875 ts_tablet_manager.cc:1428] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:09.163985 31463 heartbeater.cc:503] Master 127.24.162.190:44097 was elected leader, sending a full tablet report...
I20260808 10:03:09.164440 31873 tablet_bootstrap.cc:492] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: No bootstrap required, opened a new log
I20260808 10:03:09.164515 31873 ts_tablet_manager.cc:1397] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 10:03:09.165956 31873 raft_consensus.cc:348] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.166105 31873 raft_consensus.cc:374] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.166152 31873 raft_consensus.cc:729] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737b0afdde694426ab19448fd64fa850, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.166260 31873 consensus_queue.cc:260] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.166512 31873 ts_tablet_manager.cc:1428] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:09.166544 31645 heartbeater.cc:503] Master 127.24.162.190:44097 was elected leader, sending a full tablet report...
W20260808 10:03:09.221207 31646 tablet.cc:2369] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:03:09.314293 31465 tablet.cc:2369] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:09.366254 31883 raft_consensus.cc:482] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:09.366387 31883 raft_consensus.cc:504] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.366717 31883 leader_election.cc:288] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7415524f37894bc1bc2c0d1952804b6c (127.24.162.129:41143), b5eacef7e9fb4ffd887f90874471a733 (127.24.162.131:38679)
W20260808 10:03:09.368368 31855 tablet.cc:2369] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:09.369980 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.367210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:36815 (local address 127.24.162.131:38679)
0808 10:03:09.367269 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.367272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.367342 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:03:09.367375 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.367376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.367383 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.367454 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:09.367594 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.368280 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369251 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.369629 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369715 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.369724 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.369867 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.369904 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:03:09.370020 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.366952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41143 (local address 127.24.162.130:32935)
0808 10:03:09.367016 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.367044 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.367114 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.367284 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.367286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.367288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:09.367367 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.367368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.368291 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.368293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.369146 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.369148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.369726 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.369815 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.369818 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.369882 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.369920 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:03:09.370011 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.367008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:32935 (local address 127.24.162.129:41143)
0808 10:03:09.367075 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.367078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.367094 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:03:09.367192 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.367194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.367206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.367268 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:09.367445 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.368203 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369246 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.369611 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369729 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.369737 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.369900 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.369940 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:03:09.370038 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.367077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:38679 (local address 127.24.162.130:36815)
0808 10:03:09.367271 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.367280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.367332 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.367471 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.367472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.367473 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:09.367530 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.367531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.368348 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.368350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.369153 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:09.369156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.369714 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.369715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:09.369790 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.369793 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.369860 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.369901 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":126,"threads_started":1}
I20260808 10:03:09.370445 31787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2caf2b1e164bed8c184ad788dd469d" candidate_uuid: "737b0afdde694426ab19448fd64fa850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5eacef7e9fb4ffd887f90874471a733" is_pre_election: true
I20260808 10:03:09.370460 31397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2caf2b1e164bed8c184ad788dd469d" candidate_uuid: "737b0afdde694426ab19448fd64fa850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7415524f37894bc1bc2c0d1952804b6c" is_pre_election: true
I20260808 10:03:09.370579 31787 raft_consensus.cc:2454] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 737b0afdde694426ab19448fd64fa850 in term 0.
I20260808 10:03:09.370620 31397 raft_consensus.cc:2454] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 737b0afdde694426ab19448fd64fa850 in term 0.
I20260808 10:03:09.370843 31495 leader_election.cc:302] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 737b0afdde694426ab19448fd64fa850, b5eacef7e9fb4ffd887f90874471a733; no voters:
I20260808 10:03:09.371014 31883 raft_consensus.cc:2781] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:09.371073 31883 raft_consensus.cc:482] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:09.371102 31883 raft_consensus.cc:3037] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:09.371802 31883 raft_consensus.cc:504] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.371934 31883 leader_election.cc:288] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [CANDIDATE]: Term 1 election: Requested vote from peers 7415524f37894bc1bc2c0d1952804b6c (127.24.162.129:41143), b5eacef7e9fb4ffd887f90874471a733 (127.24.162.131:38679)
I20260808 10:03:09.372107 31397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2caf2b1e164bed8c184ad788dd469d" candidate_uuid: "737b0afdde694426ab19448fd64fa850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7415524f37894bc1bc2c0d1952804b6c"
I20260808 10:03:09.372121 31787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd2caf2b1e164bed8c184ad788dd469d" candidate_uuid: "737b0afdde694426ab19448fd64fa850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5eacef7e9fb4ffd887f90874471a733"
I20260808 10:03:09.372193 31397 raft_consensus.cc:3037] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:09.372197 31787 raft_consensus.cc:3037] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:09.373090 31397 raft_consensus.cc:2454] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 737b0afdde694426ab19448fd64fa850 in term 1.
I20260808 10:03:09.373090 31787 raft_consensus.cc:2454] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 737b0afdde694426ab19448fd64fa850 in term 1.
I20260808 10:03:09.373279 31495 leader_election.cc:302] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 737b0afdde694426ab19448fd64fa850, 7415524f37894bc1bc2c0d1952804b6c; no voters:
I20260808 10:03:09.373368 31883 raft_consensus.cc:2781] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:09.373507 31883 raft_consensus.cc:686] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 1 LEADER]: Becoming Leader. State: Replica: 737b0afdde694426ab19448fd64fa850, State: Running, Role: LEADER
I20260808 10:03:09.373615 31883 consensus_queue.cc:237] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } }
I20260808 10:03:09.374547 31180 catalog_manager.cc:5495] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 737b0afdde694426ab19448fd64fa850 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "737b0afdde694426ab19448fd64fa850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "737b0afdde694426ab19448fd64fa850" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 44303 } health_report { overall_health: HEALTHY } } }
I20260808 10:03:09.382071 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.380043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.24.162.130:44303)
0808 10:03:09.380080 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.380082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.380109 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:03:09.380166 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.380166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.380180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.380241 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:09.380357 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.380798 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.381477 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:09.381650 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:09.381711 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:09.381826 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:09.381910 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:09.381962 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.381995 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.382010 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:09.383714 31787 raft_consensus.cc:1261] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 1 FOLLOWER]: Refusing update from remote peer 737b0afdde694426ab19448fd64fa850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:09.383742 31397 raft_consensus.cc:1261] T dd2caf2b1e164bed8c184ad788dd469d P 7415524f37894bc1bc2c0d1952804b6c [term 1 FOLLOWER]: Refusing update from remote peer 737b0afdde694426ab19448fd64fa850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:09.384210 31885 consensus_queue.cc:1035] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7415524f37894bc1bc2c0d1952804b6c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:09.384347 31883 consensus_queue.cc:1035] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5eacef7e9fb4ffd887f90874471a733" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:09.386503 31889 mvcc.cc:205] Tried to move back new op lower bound from 7316207162913247232 to 7316207162875002880. Current Snapshot: MvccSnapshot[applied={T|T < 7316207162913247232}]
I20260808 10:03:09.387615 31891 mvcc.cc:205] Tried to move back new op lower bound from 7316207162913247232 to 7316207162875002880. Current Snapshot: MvccSnapshot[applied={T|T < 7316207162913247232}]
I20260808 10:03:09.389784 31180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:09.389876 31180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:09.390647 31180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2811fe313824d01a39859233a241620: Sending DeleteTablet for 3 replicas of tablet dd2caf2b1e164bed8c184ad788dd469d
I20260808 10:03:09.391021 31366 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2caf2b1e164bed8c184ad788dd469d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:48626
I20260808 10:03:09.391074 31755 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2caf2b1e164bed8c184ad788dd469d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:49524
I20260808 10:03:09.391150 31547 tablet_service.cc:1514] Processing DeleteTablet for tablet dd2caf2b1e164bed8c184ad788dd469d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:45574
I20260808 10:03:09.391345 31901 tablet_replica.cc:330] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: stopping tablet replica
I20260808 10:03:09.391450 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 31275
I20260808 10:03:09.391476 31901 raft_consensus.cc:2229] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:09.391593 31901 raft_consensus.cc:2258] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:09.392493 31901 ts_tablet_manager.cc:1905] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:09.394045 31901 ts_tablet_manager.cc:1918] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:09.394138 31901 log.cc:1196] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/dd2caf2b1e164bed8c184ad788dd469d
I20260808 10:03:09.394434 31901 ts_tablet_manager.cc:1939] T dd2caf2b1e164bed8c184ad788dd469d P 737b0afdde694426ab19448fd64fa850: Deleting consensus metadata
I20260808 10:03:09.394842 31164 catalog_manager.cc:4841] TS 737b0afdde694426ab19448fd64fa850 (127.24.162.130:44303): tablet dd2caf2b1e164bed8c184ad788dd469d (table test-table [id=183cd34f62ba4e04abdc0d03a132b667]) successfully deleted
I20260808 10:03:09.402290 31900 tablet_replica.cc:330] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: stopping tablet replica
I20260808 10:03:09.402486 31900 raft_consensus.cc:2229] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:09.402657 31900 raft_consensus.cc:2258] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:09.403950 31900 ts_tablet_manager.cc:1905] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:09.405905 31900 ts_tablet_manager.cc:1918] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:09.405987 31900 log.cc:1196] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/dd2caf2b1e164bed8c184ad788dd469d
I20260808 10:03:09.406337 31900 ts_tablet_manager.cc:1939] T dd2caf2b1e164bed8c184ad788dd469d P b5eacef7e9fb4ffd887f90874471a733: Deleting consensus metadata
I20260808 10:03:09.406808 31166 catalog_manager.cc:4841] TS b5eacef7e9fb4ffd887f90874471a733 (127.24.162.131:38679): tablet dd2caf2b1e164bed8c184ad788dd469d (table test-table [id=183cd34f62ba4e04abdc0d03a132b667]) successfully deleted
W20260808 10:03:09.408360 31166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:41143 (error 108)
W20260808 10:03:09.408872 31867 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:09.408726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41143 (local address 127.0.0.1:48642)
0808 10:03:09.408785 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.408830 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41143: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:03:09.409014 31166 catalog_manager.cc:4584] TS 7415524f37894bc1bc2c0d1952804b6c (127.24.162.129:41143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd2caf2b1e164bed8c184ad788dd469d: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41143: connect: Connection reset by peer (error 104)
I20260808 10:03:09.409325 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 31473
I20260808 10:03:09.420953 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 31656
I20260808 10:03:09.431203 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 31143
2026-08-08T10:03:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2314 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T10:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:09Z Disabled control of system clock
I20260808 10:03:09.465102 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36397
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:03:09.546552 31908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:09.546752 31908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:09.546774 31908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:09.547933 31908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:09.547977 31908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:09.547991 31908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:09.548004 31908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:09.548017 31908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:09.549445 31908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:09.549705 31908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:09.550001 31908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:09.554934 31916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.554934 31919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.555119 31908 server_base.cc:1034] running on GCE node
W20260808 10:03:09.555017 31913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.555451 31908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:09.555771 31908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:09.556945 31908 hybrid_clock.cc:648] HybridClock initialized: now 1786183389556931 us; error 49 us; skew 500 ppm
I20260808 10:03:09.561929 31908 webserver.cc:460] Webserver started at http://127.24.162.190:33751/ using document root <none> and password file <none>
I20260808 10:03:09.562175 31908 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:09.562243 31908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:09.562403 31908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:09.563536 31908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "349afd9e07fd415a8eb371300f2e5430"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "844ee7661bdca9f11fe6661468d15ea4"
server_key_iv: "bba293f7b55124d272968c777d052cd0"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.563911 31908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "349afd9e07fd415a8eb371300f2e5430"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "844ee7661bdca9f11fe6661468d15ea4"
server_key_iv: "bba293f7b55124d272968c777d052cd0"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.565229 31908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:09.566040 31926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.566241 31908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:09.566340 31908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "349afd9e07fd415a8eb371300f2e5430"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "844ee7661bdca9f11fe6661468d15ea4"
server_key_iv: "bba293f7b55124d272968c777d052cd0"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.566416 31908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:09.569850 31908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:09.570128 31908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:09.570267 31908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:09.574440 31908 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36397
I20260808 10:03:09.574470 32003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36397 every 8 connection(s)
I20260808 10:03:09.574790 31908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:09.575529 32006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.578358 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: Bootstrap starting.
I20260808 10:03:09.578984 32006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.579352 32006 log.cc:824] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.579412 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 31908
I20260808 10:03:09.579499 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:09.579689 25226 external_mini_cluster.cc:1428] Setting key ae64cd4c31f683db35cc4c3e42fb748e
I20260808 10:03:09.580180 32006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: No bootstrap required, opened a new log
I20260808 10:03:09.581872 32006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } }
I20260808 10:03:09.582034 32006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.582077 32006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349afd9e07fd415a8eb371300f2e5430, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.582597 32006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } }
I20260808 10:03:09.582736 32006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:09.582777 32006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:09.582829 32006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:09.583482 32006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } }
I20260808 10:03:09.583571 32006 leader_election.cc:302] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349afd9e07fd415a8eb371300f2e5430; no voters:
I20260808 10:03:09.583658 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.580214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.24.162.190:36397)
0808 10:03:09.580395 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.580402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.580426 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:03:09.581862 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.581869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.581907 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.582021 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.582779 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.582782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.582785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.582794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.582862 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.582879 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.582988 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.583069 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.583137 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260808 10:03:09.583746 32006 leader_election.cc:288] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:09.583900 32014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:09.583984 32006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:09.584074 32014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [term 1 LEADER]: Becoming Leader. State: Replica: 349afd9e07fd415a8eb371300f2e5430, State: Running, Role: LEADER
I20260808 10:03:09.584163 32014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } }
I20260808 10:03:09.584492 32014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349afd9e07fd415a8eb371300f2e5430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } } }
I20260808 10:03:09.584658 32014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:09.584853 32014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349afd9e07fd415a8eb371300f2e5430. Latest consensus state: current_term: 1 leader_uuid: "349afd9e07fd415a8eb371300f2e5430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349afd9e07fd415a8eb371300f2e5430" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36397 } } }
I20260808 10:03:09.584944 32014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:09.584928 32023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:09.585633 32023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:09.587213 32023 catalog_manager.cc:1347] Generated new cluster ID: b8796522910145989e9d171d7950c5ed
I20260808 10:03:09.587285 32023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:09.602905 32023 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:09.603096 32023 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:09.609582 32023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: Generated new TSK 0
I20260808 10:03:09.609814 32023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:09.617519 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--builtin_ntp_servers=127.24.162.148:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:03:09.701371 32038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:09.701567 32038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:09.701589 32038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:09.702785 32038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:09.702823 32038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:09.702850 32038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:09.704246 32038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:09.704474 32038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:09.704779 32038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:09.710511 32038 server_base.cc:1034] running on GCE node
W20260808 10:03:09.710484 32044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.710512 32045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.710484 32049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.711055 32038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:09.711270 32038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:09.712399 32038 hybrid_clock.cc:648] HybridClock initialized: now 1786183389712384 us; error 32 us; skew 500 ppm
I20260808 10:03:09.713629 32038 webserver.cc:460] Webserver started at http://127.24.162.129:40067/ using document root <none> and password file <none>
I20260808 10:03:09.713872 32038 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:09.713918 32038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:09.714083 32038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:09.715217 32038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "db078fb044b54fc884a8b292df522e29"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ab8801e6c17fca583849dd5324dda2d1"
server_key_iv: "68522967a11e27097e2dc8efa71aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.715580 32038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "db078fb044b54fc884a8b292df522e29"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ab8801e6c17fca583849dd5324dda2d1"
server_key_iv: "68522967a11e27097e2dc8efa71aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.716909 32038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:09.717593 32057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.717785 32038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:09.717898 32038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "db078fb044b54fc884a8b292df522e29"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ab8801e6c17fca583849dd5324dda2d1"
server_key_iv: "68522967a11e27097e2dc8efa71aa4ae"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.717974 32038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:09.721279 32038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:09.721537 32038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:09.721688 32038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:09.721906 32038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:09.722218 32038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:09.722271 32038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.722312 32038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:09.722343 32038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.727892 32038 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:44539
I20260808 10:03:09.727977 32216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:44539 every 8 connection(s)
I20260808 10:03:09.728295 32038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:09.731637 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.729048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:37447 (local address 127.24.162.190:36397)
0808 10:03:09.729122 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.729124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.729364 (+ 240us) server_negotiation.cc:407] Connection header received
0808 10:03:09.729894 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.729896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.729903 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.729955 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.731370 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.731373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.731373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.731378 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.731395 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.731400 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.731456 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.731549 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.731566 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:03:09.731844 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.729019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36397 (local address 127.24.162.129:37447)
0808 10:03:09.729235 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.729269 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.729765 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.730034 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.730037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.730045 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.731210 (+ 1165us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.731211 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.731242 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.731269 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.731450 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.731455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.731459 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.731558 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.731618 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260808 10:03:09.732395 32217 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36397
I20260808 10:03:09.732472 32217 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:09.732582 32217 heartbeater.cc:511] Master 127.24.162.190:36397 requested a full tablet report, sending...
I20260808 10:03:09.733107 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 32038
I20260808 10:03:09.733145 31948 ts_manager.cc:194] Registered new tserver with Master: db078fb044b54fc884a8b292df522e29 (127.24.162.129:44539)
I20260808 10:03:09.733214 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:09.733430 25226 external_mini_cluster.cc:1428] Setting key 81a22bcceb55e0721263f7790ef788fb
I20260808 10:03:09.734310 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--builtin_ntp_servers=127.24.162.148:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:03:09.816351 32222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:09.816545 32222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:09.816565 32222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:09.817699 32222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:09.817739 32222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:09.817764 32222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:09.819188 32222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:09.819438 32222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:09.819746 32222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:09.825060 32233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.825053 32237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.825340 32222 server_base.cc:1034] running on GCE node
W20260808 10:03:09.825102 32231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.825621 32222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:09.825817 32222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:09.826973 32222 hybrid_clock.cc:648] HybridClock initialized: now 1786183389826955 us; error 43 us; skew 500 ppm
I20260808 10:03:09.828337 32222 webserver.cc:460] Webserver started at http://127.24.162.130:33349/ using document root <none> and password file <none>
I20260808 10:03:09.828600 32222 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:09.828647 32222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:09.828800 32222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:09.829877 32222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "56e5bb3ff0904e37b469d40fc85a684f"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "c9386eb460b52c37e1415917ec846e80"
server_key_iv: "1f8d9b87e5067353046ab87de892ec2e"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.830253 32222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "56e5bb3ff0904e37b469d40fc85a684f"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "c9386eb460b52c37e1415917ec846e80"
server_key_iv: "1f8d9b87e5067353046ab87de892ec2e"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.831655 32222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:09.832404 32244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.832584 32222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:09.832679 32222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "56e5bb3ff0904e37b469d40fc85a684f"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "c9386eb460b52c37e1415917ec846e80"
server_key_iv: "1f8d9b87e5067353046ab87de892ec2e"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.832736 32222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:09.836061 32222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:09.836279 32222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:09.836374 32222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:09.836519 32222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:09.836761 32222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:09.836793 32222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.836812 32222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:09.836825 32222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.842417 32222 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40463
I20260808 10:03:09.842484 32394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40463 every 8 connection(s)
I20260808 10:03:09.842808 32222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:09.846055 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.843581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:41751 (local address 127.24.162.190:36397)
0808 10:03:09.843663 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.843666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.844037 (+ 371us) server_negotiation.cc:407] Connection header received
0808 10:03:09.844468 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.844469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.844475 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.844533 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.845811 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.845813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.845813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.845817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.845833 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.845837 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.845893 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.845986 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.845999 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:03:09.846226 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.843538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36397 (local address 127.24.162.130:41751)
0808 10:03:09.843879 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.843918 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.844375 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.844582 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.844585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.844600 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.845626 (+ 1026us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.845626 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.845653 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.845679 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.845912 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.845917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.845919 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.845979 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.846027 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260808 10:03:09.846664 32395 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36397
I20260808 10:03:09.846803 32395 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:09.846966 32395 heartbeater.cc:511] Master 127.24.162.190:36397 requested a full tablet report, sending...
I20260808 10:03:09.847360 31948 ts_manager.cc:194] Registered new tserver with Master: 56e5bb3ff0904e37b469d40fc85a684f (127.24.162.130:40463)
I20260808 10:03:09.849349 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 32222
I20260808 10:03:09.849462 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:09.849656 25226 external_mini_cluster.cc:1428] Setting key e312449e4a9f061dcb6b733dc6ae44aa
I20260808 10:03:09.850620 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--builtin_ntp_servers=127.24.162.148:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:03:09.935492 32398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:09.935726 32398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:09.935766 32398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:09.936978 32398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:09.937043 32398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:09.937088 32398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:09.938457 32398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:09.938728 32398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:09.939102 32398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:09.944161 32409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.944183 32405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:09.944162 32406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:09.944414 32398 server_base.cc:1034] running on GCE node
I20260808 10:03:09.944622 32398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:09.944831 32398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:09.945968 32398 hybrid_clock.cc:648] HybridClock initialized: now 1786183389945953 us; error 30 us; skew 500 ppm
I20260808 10:03:09.947309 32398 webserver.cc:460] Webserver started at http://127.24.162.131:40687/ using document root <none> and password file <none>
I20260808 10:03:09.947569 32398 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:09.947618 32398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:09.947793 32398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:09.948824 32398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "28a10216821e4732a2d06903c694bf9c"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ccb2733f43c097b34f44c1f18ac09fdd"
server_key_iv: "2190ca3def6be94b6fd347167aa38722"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.949189 32398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "28a10216821e4732a2d06903c694bf9c"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ccb2733f43c097b34f44c1f18ac09fdd"
server_key_iv: "2190ca3def6be94b6fd347167aa38722"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.950704 32398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:09.951529 32416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.951710 32398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:09.951848 32398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "28a10216821e4732a2d06903c694bf9c"
format_stamp: "Formatted at 2026-08-08 10:03:09 on dist-test-slave-nfb5"
server_key: "ccb2733f43c097b34f44c1f18ac09fdd"
server_key_iv: "2190ca3def6be94b6fd347167aa38722"
server_key_version: "encryptionkey@0"
I20260808 10:03:09.951906 32398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:09.955243 32398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:09.955471 32398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:09.955608 32398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:09.955811 32398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:09.956115 32398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:09.956153 32398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.956213 32398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:09.956243 32398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:09.962188 32398 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:36903
I20260808 10:03:09.962221 32580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:36903 every 8 connection(s)
I20260808 10:03:09.962554 32398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:09.965415 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.963318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:39189 (local address 127.24.162.190:36397)
0808 10:03:09.963406 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.963409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.963610 (+ 201us) server_negotiation.cc:407] Connection header received
0808 10:03:09.964029 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.964030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.964038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.964092 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.965205 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.965207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.965207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.965211 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.965226 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.965231 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.965283 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.965347 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.965361 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:03:09.965590 32584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.963276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36397 (local address 127.24.162.131:39189)
0808 10:03:09.963481 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.963514 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.963948 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.964139 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.964142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.964150 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.965045 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.965045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.965072 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.965098 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.965279 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.965281 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.965284 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.965352 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.965397 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260808 10:03:09.965899 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 32398
I20260808 10:03:09.966019 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:09.966259 25226 external_mini_cluster.cc:1428] Setting key e698591569eabd99656eebdba0eab5f7
I20260808 10:03:09.966287 32581 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36397
I20260808 10:03:09.966392 32581 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:09.966563 32581 heartbeater.cc:511] Master 127.24.162.190:36397 requested a full tablet report, sending...
I20260808 10:03:09.966971 31947 ts_manager.cc:194] Registered new tserver with Master: 28a10216821e4732a2d06903c694bf9c (127.24.162.131:36903)
I20260808 10:03:09.967448 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:03:09.969939 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.968735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.24.162.190:36397)
0808 10:03:09.968821 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.968823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.968933 (+ 110us) server_negotiation.cc:407] Connection header received
0808 10:03:09.969011 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.969012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.969017 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.969072 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.969726 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.969727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.969728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.969732 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.969744 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.969748 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.969792 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.969856 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.969870 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:03:09.971617 31948 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:09.973090 31948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:09.976886 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40463 (local address 127.0.0.1:36530)
0808 10:03:09.975317 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.975361 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.975419 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.975619 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.975625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.975631 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.976382 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.976383 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.976399 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.976421 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.976649 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.976658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.976660 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.976775 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.976791 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":55,"threads_started":1}
I20260808 10:03:09.976926 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.24.162.130:40463)
0808 10:03:09.975467 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.975472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.975481 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:09.975505 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.975507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.975515 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.975618 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.976510 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.976518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.976518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.976523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.976555 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.976576 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.976644 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.976800 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.976828 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260808 10:03:09.977245 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:36903 (local address 127.0.0.1:38332)
0808 10:03:09.975665 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.975677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.975737 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.975959 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.975960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.975966 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.976382 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.976383 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.976399 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.976421 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.977118 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.977119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.977120 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.977176 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.977188 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":82,"threads_started":1}
I20260808 10:03:09.977510 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.24.162.131:36903)
0808 10:03:09.975658 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.975662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.975797 (+ 135us) server_negotiation.cc:407] Connection header received
0808 10:03:09.975830 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.975832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.975839 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.975951 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.976508 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.976515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.976515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.976519 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.976555 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.976575 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.976644 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.977399 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.977417 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260808 10:03:09.977561 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44539 (local address 127.0.0.1:32986)
0808 10:03:09.975711 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:09.975720 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:09.975756 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:09.976710 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:09.976711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:09.976714 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:09.977168 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:09.977169 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:09.977176 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:09.977190 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:09.977426 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:09.977427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:09.977428 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:03:09.977471 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 10:03:09.977483 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":75,"threads_started":1}
I20260808 10:03:09.977730 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:09.975483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.24.162.129:44539)
0808 10:03:09.976574 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:09.976577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:09.976585 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:09.976605 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:09.976606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:09.976632 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:09.976709 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:09.977277 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:09.977280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:09.977281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:09.977284 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:09.977315 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:09.977333 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:09.977396 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:09.977625 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:03:09.977643 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":86,"threads_started":1}
I20260808 10:03:09.977782 32302 tablet_service.cc:1467] Processing CreateTablet for tablet fb659c718a96486f837f005eb7c5aa66 (DEFAULT_TABLE table=test-table [id=b41a21873b77436d8a64930234a7aced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.978204 32302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb659c718a96486f837f005eb7c5aa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.978367 32482 tablet_service.cc:1467] Processing CreateTablet for tablet fb659c718a96486f837f005eb7c5aa66 (DEFAULT_TABLE table=test-table [id=b41a21873b77436d8a64930234a7aced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.978715 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb659c718a96486f837f005eb7c5aa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.980964 32599 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Bootstrap starting.
I20260808 10:03:09.981266 32600 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Bootstrap starting.
I20260808 10:03:09.981681 32599 tablet_bootstrap.cc:654] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.981745 32123 tablet_service.cc:1467] Processing CreateTablet for tablet fb659c718a96486f837f005eb7c5aa66 (DEFAULT_TABLE table=test-table [id=b41a21873b77436d8a64930234a7aced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:09.981967 32600 tablet_bootstrap.cc:654] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.982033 32599 log.cc:824] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.982123 32123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb659c718a96486f837f005eb7c5aa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:09.982293 32600 log.cc:824] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.982746 32599 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: No bootstrap required, opened a new log
I20260808 10:03:09.982800 32599 ts_tablet_manager.cc:1397] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:09.983574 32600 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: No bootstrap required, opened a new log
I20260808 10:03:09.983712 32600 ts_tablet_manager.cc:1397] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:09.985496 32605 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: Bootstrap starting.
I20260808 10:03:09.985430 32599 raft_consensus.cc:348] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.985611 32599 raft_consensus.cc:374] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.985658 32599 raft_consensus.cc:729] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e5bb3ff0904e37b469d40fc85a684f, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.985796 32599 consensus_queue.cc:260] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.986086 32599 ts_tablet_manager.cc:1428] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:09.986250 32395 heartbeater.cc:503] Master 127.24.162.190:36397 was elected leader, sending a full tablet report...
I20260808 10:03:09.986330 32605 tablet_bootstrap.cc:654] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:09.986486 32600 raft_consensus.cc:348] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.986629 32600 raft_consensus.cc:374] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.986671 32600 raft_consensus.cc:729] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a10216821e4732a2d06903c694bf9c, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.986670 32605 log.cc:824] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:09.986800 32600 consensus_queue.cc:260] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.987077 32600 ts_tablet_manager.cc:1428] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:03:09.987246 32581 heartbeater.cc:503] Master 127.24.162.190:36397 was elected leader, sending a full tablet report...
I20260808 10:03:09.987958 32605 tablet_bootstrap.cc:492] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: No bootstrap required, opened a new log
I20260808 10:03:09.988026 32605 ts_tablet_manager.cc:1397] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:03:09.989871 32605 raft_consensus.cc:348] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.989993 32605 raft_consensus.cc:374] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:09.990015 32605 raft_consensus.cc:729] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db078fb044b54fc884a8b292df522e29, State: Initialized, Role: FOLLOWER
I20260808 10:03:09.990120 32605 consensus_queue.cc:260] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:09.990391 32605 ts_tablet_manager.cc:1428] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:03:09.990419 32217 heartbeater.cc:503] Master 127.24.162.190:36397 was elected leader, sending a full tablet report...
I20260808 10:03:10.061306 32607 raft_consensus.cc:482] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:10.061445 32607 raft_consensus.cc:504] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:10.061837 32607 leader_election.cc:288] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28a10216821e4732a2d06903c694bf9c (127.24.162.131:36903), db078fb044b54fc884a8b292df522e29 (127.24.162.129:44539)
I20260808 10:03:10.063303 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.062086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:36903 (local address 127.24.162.130:59789)
0808 10:03:10.062156 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.062170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:10.062252 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:10.062408 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:10.062409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:10.062412 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:10.062904 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:10.062905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:10.062956 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:10.062971 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:10.063171 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:10.063172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:10.063172 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 10:03:10.063224 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:03:10.063238 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:03:10.063359 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.062167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:59789 (local address 127.24.162.131:36903)
0808 10:03:10.062248 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.062250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.062264 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:03:10.062322 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.062323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.062329 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.062396 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.063060 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.063062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.063062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.063067 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.063087 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.063091 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.063150 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.063247 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.063263 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:03:10.063490 32614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.062246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44539 (local address 127.24.162.130:33609)
0808 10:03:10.062437 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.062447 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:10.062501 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:10.062649 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:10.062650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:10.062652 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:10.063115 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:10.063115 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:10.063120 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:10.063147 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:10.063347 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:10.063347 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:10.063348 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:03:10.063401 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:03:10.063414 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":134,"threads_started":1}
I20260808 10:03:10.063949 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.062361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:33609 (local address 127.24.162.129:44539)
0808 10:03:10.062456 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.062458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.062519 (+ 61us) server_negotiation.cc:407] Connection header received
0808 10:03:10.062553 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.062554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.062558 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.062630 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.063251 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.063253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.063253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.063258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.063276 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.063281 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.063339 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.063885 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.063899 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:03:10.064070 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb659c718a96486f837f005eb7c5aa66" candidate_uuid: "56e5bb3ff0904e37b469d40fc85a684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a10216821e4732a2d06903c694bf9c" is_pre_election: true
I20260808 10:03:10.064236 32515 raft_consensus.cc:2454] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e5bb3ff0904e37b469d40fc85a684f in term 0.
I20260808 10:03:10.064312 32159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb659c718a96486f837f005eb7c5aa66" candidate_uuid: "56e5bb3ff0904e37b469d40fc85a684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db078fb044b54fc884a8b292df522e29" is_pre_election: true
I20260808 10:03:10.064486 32159 raft_consensus.cc:2454] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e5bb3ff0904e37b469d40fc85a684f in term 0.
I20260808 10:03:10.064491 32247 leader_election.cc:302] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a10216821e4732a2d06903c694bf9c, 56e5bb3ff0904e37b469d40fc85a684f; no voters:
I20260808 10:03:10.064640 32607 raft_consensus.cc:2781] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:10.064729 32607 raft_consensus.cc:482] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:10.064766 32607 raft_consensus.cc:3037] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.065487 32607 raft_consensus.cc:504] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:10.065626 32607 leader_election.cc:288] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [CANDIDATE]: Term 1 election: Requested vote from peers 28a10216821e4732a2d06903c694bf9c (127.24.162.131:36903), db078fb044b54fc884a8b292df522e29 (127.24.162.129:44539)
I20260808 10:03:10.065817 32159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb659c718a96486f837f005eb7c5aa66" candidate_uuid: "56e5bb3ff0904e37b469d40fc85a684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db078fb044b54fc884a8b292df522e29"
I20260808 10:03:10.065862 32515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb659c718a96486f837f005eb7c5aa66" candidate_uuid: "56e5bb3ff0904e37b469d40fc85a684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a10216821e4732a2d06903c694bf9c"
I20260808 10:03:10.065917 32159 raft_consensus.cc:3037] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.065953 32515 raft_consensus.cc:3037] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.066766 32515 raft_consensus.cc:2454] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e5bb3ff0904e37b469d40fc85a684f in term 1.
I20260808 10:03:10.066957 32247 leader_election.cc:302] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a10216821e4732a2d06903c694bf9c, 56e5bb3ff0904e37b469d40fc85a684f; no voters:
I20260808 10:03:10.066964 32159 raft_consensus.cc:2454] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e5bb3ff0904e37b469d40fc85a684f in term 1.
I20260808 10:03:10.067116 32607 raft_consensus.cc:2781] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:10.067266 32607 raft_consensus.cc:686] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 1 LEADER]: Becoming Leader. State: Replica: 56e5bb3ff0904e37b469d40fc85a684f, State: Running, Role: LEADER
I20260808 10:03:10.067386 32607 consensus_queue.cc:237] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } }
I20260808 10:03:10.068243 31947 catalog_manager.cc:5495] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e5bb3ff0904e37b469d40fc85a684f (127.24.162.130). New cstate: current_term: 1 leader_uuid: "56e5bb3ff0904e37b469d40fc85a684f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e5bb3ff0904e37b469d40fc85a684f" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:10.093106 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.092081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.24.162.130:40463)
0808 10:03:10.092145 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.092147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.092155 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:10.092191 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.092191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.092205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.092254 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.092885 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.092885 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.092886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.092888 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.092899 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.092902 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.092943 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.092991 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.093002 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 10:03:10.093346 32396 tablet.cc:2369] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:10.094863 32159 raft_consensus.cc:1261] T fb659c718a96486f837f005eb7c5aa66 P db078fb044b54fc884a8b292df522e29 [term 1 FOLLOWER]: Refusing update from remote peer 56e5bb3ff0904e37b469d40fc85a684f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:10.094895 32515 raft_consensus.cc:1261] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 1 FOLLOWER]: Refusing update from remote peer 56e5bb3ff0904e37b469d40fc85a684f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:10.095275 32615 consensus_queue.cc:1035] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db078fb044b54fc884a8b292df522e29" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:10.095413 32607 consensus_queue.cc:1035] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a10216821e4732a2d06903c694bf9c" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:10.098502 32622 mvcc.cc:205] Tried to move back new op lower bound from 7316207165825794048 to 7316207165716779008. Current Snapshot: MvccSnapshot[applied={T|T < 7316207165825794048}]
I20260808 10:03:10.100153 31947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:10.100251 31947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:10.101495 31947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 349afd9e07fd415a8eb371300f2e5430: Sending DeleteTablet for 3 replicas of tablet fb659c718a96486f837f005eb7c5aa66
I20260808 10:03:10.101871 32302 tablet_service.cc:1514] Processing DeleteTablet for tablet fb659c718a96486f837f005eb7c5aa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:10 UTC) from {username='slave'} at 127.0.0.1:36530
I20260808 10:03:10.101910 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet fb659c718a96486f837f005eb7c5aa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:10 UTC) from {username='slave'} at 127.0.0.1:38332
I20260808 10:03:10.101909 32123 tablet_service.cc:1514] Processing DeleteTablet for tablet fb659c718a96486f837f005eb7c5aa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:10 UTC) from {username='slave'} at 127.0.0.1:32986
I20260808 10:03:10.102108 32634 tablet_replica.cc:330] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: stopping tablet replica
I20260808 10:03:10.102197 32634 raft_consensus.cc:2229] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:10.102306 32634 raft_consensus.cc:2258] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:10.102365 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 32038
I20260808 10:03:10.102767 32634 ts_tablet_manager.cc:1905] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:10.102403 32636 tablet_replica.cc:330] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: stopping tablet replica
I20260808 10:03:10.103360 32636 raft_consensus.cc:2229] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:10.103550 32636 raft_consensus.cc:2258] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:10.105162 32634 ts_tablet_manager.cc:1918] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:10.105249 32634 log.cc:1196] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/fb659c718a96486f837f005eb7c5aa66
I20260808 10:03:10.105507 32636 ts_tablet_manager.cc:1905] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:10.105585 32634 ts_tablet_manager.cc:1939] T fb659c718a96486f837f005eb7c5aa66 P 56e5bb3ff0904e37b469d40fc85a684f: Deleting consensus metadata
I20260808 10:03:10.106027 31930 catalog_manager.cc:4841] TS 56e5bb3ff0904e37b469d40fc85a684f (127.24.162.130:40463): tablet fb659c718a96486f837f005eb7c5aa66 (table test-table [id=b41a21873b77436d8a64930234a7aced]) successfully deleted
I20260808 10:03:10.108068 32636 ts_tablet_manager.cc:1918] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:10.108206 32636 log.cc:1196] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/fb659c718a96486f837f005eb7c5aa66
I20260808 10:03:10.108517 32636 ts_tablet_manager.cc:1939] T fb659c718a96486f837f005eb7c5aa66 P 28a10216821e4732a2d06903c694bf9c: Deleting consensus metadata
I20260808 10:03:10.108925 31929 catalog_manager.cc:4841] TS 28a10216821e4732a2d06903c694bf9c (127.24.162.131:36903): tablet fb659c718a96486f837f005eb7c5aa66 (table test-table [id=b41a21873b77436d8a64930234a7aced]) successfully deleted
W20260808 10:03:10.117640 31929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:44539 (error 108)
I20260808 10:03:10.117821 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 32222
I20260808 10:03:10.118243 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.118117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44539 (local address 127.0.0.1:33002)
0808 10:03:10.118164 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.118202 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:03:10.118352 31929 catalog_manager.cc:4584] TS db078fb044b54fc884a8b292df522e29 (127.24.162.129:44539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb659c718a96486f837f005eb7c5aa66: Network error: Client connection negotiation failed: client connection to 127.24.162.129:44539: connect: Connection refused (error 111)
I20260808 10:03:10.126387 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 32398
I20260808 10:03:10.134755 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 31908
2026-08-08T10:03:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (702 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 10:03:10.159942 25226 mini_oidc.cc:152] Starting JWKS server
I20260808 10:03:10.161099 25226 webserver.cc:460] Webserver started at https://127.0.0.1:41315/ using document root <none> and password file <none>
I20260808 10:03:10.161180 25226 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:03:10.161326 25226 webserver.cc:460] Webserver started at http://127.0.0.1:40503/ using document root <none> and password file <none>
2026-08-08T10:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:10Z Disabled control of system clock
I20260808 10:03:10.170267 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33977
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33977
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41315/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/testchainca.cert with env {}
W20260808 10:03:10.253235 32645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:10.253443 32645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:10.253466 32645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:10.254637 32645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:10.254679 32645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:10.254693 32645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:10.254705 32645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:10.254717 32645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:03:10.254729 32645 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:03:10.254741 32645 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41315/jwks.json
I20260808 10:03:10.256263 32645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41315/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:10.256451 32645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:10.256611 32645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:10.261549 32655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.261587 32658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.261587 32654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.261898 32645 server_base.cc:1034] running on GCE node
I20260808 10:03:10.262127 32645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:10.262370 32645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:10.263541 32645 hybrid_clock.cc:648] HybridClock initialized: now 1786183390263523 us; error 31 us; skew 500 ppm
I20260808 10:03:10.264845 32645 webserver.cc:460] Webserver started at http://127.24.162.190:42725/ using document root <none> and password file <none>
I20260808 10:03:10.265079 32645 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:10.265147 32645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:10.265324 32645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:10.266440 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "fcca55e3f488484faaac418d0130415a"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "d7f5cc2acfcdf3c1c494c71ae07e8e17"
server_key_iv: "4391e8535e79bd609466a66b23fa60a6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.266856 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "fcca55e3f488484faaac418d0130415a"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "d7f5cc2acfcdf3c1c494c71ae07e8e17"
server_key_iv: "4391e8535e79bd609466a66b23fa60a6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.268316 32645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:10.269021 32666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:10.269207 32645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:10.269317 32645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "fcca55e3f488484faaac418d0130415a"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "d7f5cc2acfcdf3c1c494c71ae07e8e17"
server_key_iv: "4391e8535e79bd609466a66b23fa60a6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.269390 32645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:10.287966 32645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:10.288322 32645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:10.288494 32645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:10.292798 32645 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33977
I20260808 10:03:10.292834 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33977 every 8 connection(s)
I20260808 10:03:10.293210 32645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:10.294250 32739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:10.294824 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 32645
I20260808 10:03:10.294984 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:10.295163 25226 external_mini_cluster.cc:1428] Setting key fddfe600e5e7d9ebeebeed30ca54a43d
I20260808 10:03:10.297185 32739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a: Bootstrap starting.
I20260808 10:03:10.297948 32739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:10.298286 32739 log.cc:824] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:10.299152 32739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a: No bootstrap required, opened a new log
I20260808 10:03:10.300808 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.295682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.24.162.190:33977)
0808 10:03:10.295866 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.295873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.295897 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:03:10.296342 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.296349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.296409 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.296519 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.296753 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.297671 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.298789 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.299321 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.299417 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.300095 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.300098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.300098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.300279 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.300311 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.300322 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.300375 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.300488 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.300569 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 10:03:10.301430 32739 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } }
I20260808 10:03:10.301584 32739 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:10.301633 32739 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca55e3f488484faaac418d0130415a, State: Initialized, Role: FOLLOWER
I20260808 10:03:10.301759 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:03:10.301771 32739 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } }
I20260808 10:03:10.301869 32739 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:10.301909 32739 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:10.301962 32739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.302476 32739 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } }
I20260808 10:03:10.302589 32739 leader_election.cc:302] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcca55e3f488484faaac418d0130415a; no voters:
I20260808 10:03:10.302788 32739 leader_election.cc:288] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:10.302810 32747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:10.302946 32747 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [term 1 LEADER]: Becoming Leader. State: Replica: fcca55e3f488484faaac418d0130415a, State: Running, Role: LEADER
I20260808 10:03:10.303035 32747 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } }
I20260808 10:03:10.303081 32739 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:10.303409 32750 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcca55e3f488484faaac418d0130415a. Latest consensus state: current_term: 1 leader_uuid: "fcca55e3f488484faaac418d0130415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } } }
I20260808 10:03:10.303597 32750 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.303417 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcca55e3f488484faaac418d0130415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcca55e3f488484faaac418d0130415a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33977 } } }
I20260808 10:03:10.303838 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.304061 32759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:10.304646 32759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:10.306119 32759 catalog_manager.cc:1347] Generated new cluster ID: dfcb00b707ab406ca08c8565e1b52c46
I20260808 10:03:10.306180 32759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:10.310128 32759 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:10.310619 32759 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:10.321238 32759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcca55e3f488484faaac418d0130415a: Generated new TSK 0
I20260808 10:03:10.321413 32759 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:03:10.344111 312 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 10:03:10.344269 312 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:03:10.344461 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.328727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.24.162.190:33977)
0808 10:03:10.328773 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.328776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.328927 (+ 151us) server_negotiation.cc:407] Connection header received
0808 10:03:10.328989 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.328990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.329003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.329088 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:03:10.329226 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.330007 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.330727 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.331105 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.331177 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.331360 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:03:10.344225 (+ 12865us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:03:10.344385 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54876: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:03:10.344545 32743 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54876: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 10:03:10.344971 311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:10.328642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33977 (local address 127.0.0.1:54876)
0808 10:03:10.328857 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.328867 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:10.328922 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:10.329099 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:10.329100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:10.329101 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:03:10.329163 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.329164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.330094 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.330094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.330644 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.330645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.331186 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.331187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.331240 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.331258 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:03:10.344441 (+ 13183us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:03:10.344577 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.162.190:33977: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.162.190:33977: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 10:03:10.345544 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 32645
2026-08-08T10:03:10Z chronyd exiting
I20260808 10:03:10.394318 25226 test_util.cc:175] -----------------------------------------------
I20260808 10:03:10.394402 25226 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183366714887-25226-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 10:03:10.395432 25226 mini_oidc.cc:152] Starting JWKS server
I20260808 10:03:10.396153 25226 webserver.cc:460] Webserver started at https://127.0.0.1:37323/ using document root <none> and password file <none>
I20260808 10:03:10.396227 25226 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:03:10.396376 25226 webserver.cc:460] Webserver started at http://127.0.0.1:40329/ using document root <none> and password file <none>
2026-08-08T10:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:10Z Disabled control of system clock
I20260808 10:03:10.405365 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44595
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:44595
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37323/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/testchainca.cert with env {}
W20260808 10:03:10.487738 320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:10.487962 320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:10.487985 320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:10.489264 320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:10.489332 320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:10.489360 320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:10.489375 320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:10.489388 320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:03:10.489423 320 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:03:10.489451 320 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37323/jwks.json
I20260808 10:03:10.491055 320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:44595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37323/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:10.491331 320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:10.491566 320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:10.496488 326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.496532 327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.496774 320 server_base.cc:1034] running on GCE node
W20260808 10:03:10.496783 329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.497093 320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:10.497390 320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:10.498571 320 hybrid_clock.cc:648] HybridClock initialized: now 1786183390498570 us; error 38 us; skew 500 ppm
I20260808 10:03:10.499938 320 webserver.cc:460] Webserver started at http://127.24.162.190:46305/ using document root <none> and password file <none>
I20260808 10:03:10.500185 320 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:10.500272 320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:10.500423 320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:10.501510 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "b21321fda17744959080dfafa3d9db0d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "9724d91d6e6092bf6c11b437e70a9b5f"
server_key_iv: "af839b45b625c39893dba9dcae280040"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.501950 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "b21321fda17744959080dfafa3d9db0d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "9724d91d6e6092bf6c11b437e70a9b5f"
server_key_iv: "af839b45b625c39893dba9dcae280040"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.503384 320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:10.504081 338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:10.504258 320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:10.504364 320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "b21321fda17744959080dfafa3d9db0d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "9724d91d6e6092bf6c11b437e70a9b5f"
server_key_iv: "af839b45b625c39893dba9dcae280040"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.504443 320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:10.539603 320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:10.540030 320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:10.540201 320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:10.544613 320 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:44595
I20260808 10:03:10.544653 412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:44595 every 8 connection(s)
I20260808 10:03:10.545091 320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:10.545792 414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:10.548601 414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d: Bootstrap starting.
I20260808 10:03:10.549197 414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:10.549451 414 log.cc:824] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:10.550107 414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d: No bootstrap required, opened a new log
I20260808 10:03:10.550191 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 320
I20260808 10:03:10.550339 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:10.550534 25226 external_mini_cluster.cc:1428] Setting key bd0ef337444ab895463b9e1dcd20b175
I20260808 10:03:10.552616 414 raft_consensus.cc:348] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } }
I20260808 10:03:10.552805 414 raft_consensus.cc:374] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:10.552856 414 raft_consensus.cc:729] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21321fda17744959080dfafa3d9db0d, State: Initialized, Role: FOLLOWER
I20260808 10:03:10.553013 414 consensus_queue.cc:260] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } }
I20260808 10:03:10.553102 414 raft_consensus.cc:388] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:10.553131 414 raft_consensus.cc:482] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:10.553177 414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.553887 414 raft_consensus.cc:504] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } }
I20260808 10:03:10.553997 414 leader_election.cc:302] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b21321fda17744959080dfafa3d9db0d; no voters:
I20260808 10:03:10.554140 414 leader_election.cc:288] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:10.554191 422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:10.554369 422 raft_consensus.cc:686] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [term 1 LEADER]: Becoming Leader. State: Replica: b21321fda17744959080dfafa3d9db0d, State: Running, Role: LEADER
I20260808 10:03:10.554373 414 sys_catalog.cc:564] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:10.554484 422 consensus_queue.cc:237] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } }
I20260808 10:03:10.554857 422 sys_catalog.cc:455] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b21321fda17744959080dfafa3d9db0d. Latest consensus state: current_term: 1 leader_uuid: "b21321fda17744959080dfafa3d9db0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } } }
I20260808 10:03:10.554935 422 sys_catalog.cc:458] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.555124 423 sys_catalog.cc:455] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b21321fda17744959080dfafa3d9db0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b21321fda17744959080dfafa3d9db0d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44595 } } }
I20260808 10:03:10.555202 423 sys_catalog.cc:458] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.555344 432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:10.555891 432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:10.557616 432 catalog_manager.cc:1347] Generated new cluster ID: 1d76fa72f20b4ee298f5c71783ad1450
I20260808 10:03:10.557672 432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:10.558130 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.551102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.24.162.190:44595)
0808 10:03:10.551369 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.551379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.551413 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:03:10.552157 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.552168 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.552249 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.552424 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.552611 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.554354 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.555538 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.556201 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.556384 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.557259 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.557264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.557264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.557469 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.557512 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.557526 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.557627 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.557835 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.557918 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":136,"threads_started":1}
I20260808 10:03:10.559199 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:03:10.568289 432 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:10.568852 432 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:10.572631 432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b21321fda17744959080dfafa3d9db0d: Generated new TSK 0
I20260808 10:03:10.572824 432 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:03:10.593325 459 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 10:03:10.593413 459 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:03:10.593783 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.579705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.24.162.190:44595)
0808 10:03:10.579770 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.579772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.579882 (+ 110us) server_negotiation.cc:407] Connection header received
0808 10:03:10.579961 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.579962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.579974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.580045 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:03:10.580165 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.580673 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.581457 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.581679 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.581755 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.581949 (+ 194us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:03:10.593544 (+ 11595us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:03:10.593703 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:03:10.593845 421 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 10:03:10.593871 458 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:10.579645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:44595 (local address 127.0.0.1:59632)
0808 10:03:10.579811 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.579820 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:10.579899 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:10.580044 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:10.580045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:10.580047 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:03:10.580101 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.580103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.580768 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.580769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.581369 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.581371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.581760 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.581761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.581831 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.581842 (+ 11us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:03:10.593652 (+ 11810us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:03:10.593770 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.162.190:44595: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260808 10:03:10.594095 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 320
2026-08-08T10:03:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 10:03:10.638839 25226 mini_oidc.cc:152] Starting JWKS server
I20260808 10:03:10.639559 25226 webserver.cc:460] Webserver started at https://127.0.0.1:36475/ using document root <none> and password file <none>
I20260808 10:03:10.639626 25226 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:03:10.639787 25226 webserver.cc:460] Webserver started at http://127.0.0.1:45681/ using document root <none> and password file <none>
2026-08-08T10:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:10Z Disabled control of system clock
I20260808 10:03:10.648972 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34995
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:34995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36475/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:03:10.731629 473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:10.731848 473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:10.731870 473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:10.733129 473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:10.733170 473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:10.733184 473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:10.733229 473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:10.733256 473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:03:10.733283 473 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:03:10.733309 473 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36475/jwks.json
I20260808 10:03:10.734733 473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:34995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36475/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:10.735042 473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:10.735376 473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:10.740367 482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.740428 479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.740382 480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.740664 473 server_base.cc:1034] running on GCE node
I20260808 10:03:10.740929 473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:10.741225 473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:10.742408 473 hybrid_clock.cc:648] HybridClock initialized: now 1786183390742386 us; error 38 us; skew 500 ppm
I20260808 10:03:10.743808 473 webserver.cc:460] Webserver started at http://127.24.162.190:42165/ using document root <none> and password file <none>
I20260808 10:03:10.744077 473 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:10.744128 473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:10.744292 473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:10.745507 473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "4cc79fe2d6524ec7b903f8d5f962c87f"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "241a7b381b48ca2aad2c5f5bb658cd65"
server_key_iv: "a56fa5b8b5f65646befbeb33dbb133e6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.745916 473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "4cc79fe2d6524ec7b903f8d5f962c87f"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "241a7b381b48ca2aad2c5f5bb658cd65"
server_key_iv: "a56fa5b8b5f65646befbeb33dbb133e6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.747488 473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:10.748310 491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:10.748538 473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:10.748643 473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "4cc79fe2d6524ec7b903f8d5f962c87f"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "241a7b381b48ca2aad2c5f5bb658cd65"
server_key_iv: "a56fa5b8b5f65646befbeb33dbb133e6"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.748731 473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:10.759124 473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:10.759441 473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:10.759593 473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:10.763929 473 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:34995
I20260808 10:03:10.763963 563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:34995 every 8 connection(s)
I20260808 10:03:10.764307 473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:10.765021 564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:10.767832 564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: Bootstrap starting.
I20260808 10:03:10.768486 564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:10.768779 564 log.cc:824] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:10.769476 564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: No bootstrap required, opened a new log
I20260808 10:03:10.771142 564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } }
I20260808 10:03:10.771287 564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:10.771310 564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc79fe2d6524ec7b903f8d5f962c87f, State: Initialized, Role: FOLLOWER
I20260808 10:03:10.771498 564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } }
I20260808 10:03:10.771560 564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:10.771600 564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:10.771649 564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.772197 564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } }
I20260808 10:03:10.772312 564 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc79fe2d6524ec7b903f8d5f962c87f; no voters:
I20260808 10:03:10.772508 564 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:10.772562 568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:10.772722 568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [term 1 LEADER]: Becoming Leader. State: Replica: 4cc79fe2d6524ec7b903f8d5f962c87f, State: Running, Role: LEADER
I20260808 10:03:10.772763 564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:10.772815 568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } }
I20260808 10:03:10.773203 571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc79fe2d6524ec7b903f8d5f962c87f. Latest consensus state: current_term: 1 leader_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } } }
I20260808 10:03:10.773360 571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.773486 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 473
I20260808 10:03:10.773600 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:10.773805 25226 external_mini_cluster.cc:1428] Setting key 0e3051123162e000870675719c72e74f
I20260808 10:03:10.774082 569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc79fe2d6524ec7b903f8d5f962c87f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34995 } } }
I20260808 10:03:10.774164 569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f [sys.catalog]: This master's current role is: LEADER
W20260808 10:03:10.774641 590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:10.774700 590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:10.774761 583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:10.775003 583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:10.777021 583 catalog_manager.cc:1347] Generated new cluster ID: 1fb1dd36a5724af98e18ce7e9c6aedb1
I20260808 10:03:10.777077 583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:10.779845 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.774281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.24.162.190:34995)
0808 10:03:10.774566 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.774575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.774610 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:03:10.774946 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.774954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.775023 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.775190 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.775377 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.776541 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.777665 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.778329 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.778459 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.779127 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.779131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.779132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.779343 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.779386 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.779400 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.779477 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.779545 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.779621 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
I20260808 10:03:10.780839 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:03:10.783674 583 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:10.784147 583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:10.789268 583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cc79fe2d6524ec7b903f8d5f962c87f: Generated new TSK 0
I20260808 10:03:10.789443 583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:10.802498 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.797666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.24.162.190:34995)
0808 10:03:10.797710 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.797712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.798223 (+ 511us) server_negotiation.cc:407] Connection header received
0808 10:03:10.798540 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.798541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.798560 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.798636 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:03:10.798847 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.799654 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.800598 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.800949 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.800995 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.801289 (+ 294us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:03:10.802257 (+ 968us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:03:10.802397 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49850: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:03:10.802557 592 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49850: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 10:03:10.804487 602 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:10.797653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34995 (local address 127.0.0.1:49850)
0808 10:03:10.798150 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:10.798162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:10.798252 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:10.798687 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:10.798688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:10.798690 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:03:10.798769 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.798771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.799770 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.799772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.800464 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:10.800467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.801079 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.801080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:10.801167 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.801172 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:03:10.802360 (+ 1188us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:03:10.804383 (+ 2023us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.162.190:34995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":1852,"thread_start_us":90,"threads_started":1}
I20260808 10:03:10.804754 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 473
E20260808 10:03:10.819870 603 webserver.cc:533] Webserver: sslize failed:
E20260808 10:03:10.819941 603 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T10:03:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (212 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T10:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:10Z Disabled control of system clock
I20260808 10:03:10.861060 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35199
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:35199
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/testchainca.cert with env {}
W20260808 10:03:10.944911 607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:10.945204 607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:10.945250 607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:10.946455 607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:10.946527 607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:10.946550 607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:10.946570 607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:10.946602 607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:03:10.946625 607 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:03:10.946662 607 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 10:03:10.948244 607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:35199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:10.948536 607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:10.948771 607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:10.953763 615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:10.953840 614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.953928 607 server_base.cc:1034] running on GCE node
W20260808 10:03:10.953758 617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:10.954281 607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:10.954546 607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:10.955724 607 hybrid_clock.cc:648] HybridClock initialized: now 1786183390955700 us; error 45 us; skew 500 ppm
I20260808 10:03:10.957110 607 webserver.cc:460] Webserver started at http://127.24.162.190:36053/ using document root <none> and password file <none>
I20260808 10:03:10.957370 607 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:10.957419 607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:10.957589 607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:10.958705 607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "2857c8d54468435184838999a0d0f77d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "e1fee4789427bbbc9da24f37d4f95b61"
server_key_iv: "eaa21a0fd659dc206cc139ae21ec4061"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.959115 607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "2857c8d54468435184838999a0d0f77d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "e1fee4789427bbbc9da24f37d4f95b61"
server_key_iv: "eaa21a0fd659dc206cc139ae21ec4061"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.960529 607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:10.961306 626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:10.961499 607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:10.961594 607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "2857c8d54468435184838999a0d0f77d"
format_stamp: "Formatted at 2026-08-08 10:03:10 on dist-test-slave-nfb5"
server_key: "e1fee4789427bbbc9da24f37d4f95b61"
server_key_iv: "eaa21a0fd659dc206cc139ae21ec4061"
server_key_version: "encryptionkey@0"
I20260808 10:03:10.961666 607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:10.980142 607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:10.980460 607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:10.980608 607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:10.984733 607 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:35199
I20260808 10:03:10.984790 708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:35199 every 8 connection(s)
I20260808 10:03:10.985123 607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:10.986133 709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:10.986498 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 607
I20260808 10:03:10.986629 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:10.986853 25226 external_mini_cluster.cc:1428] Setting key cbd4ce52be0d9196b788651dfed3714b
I20260808 10:03:10.989500 709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d: Bootstrap starting.
I20260808 10:03:10.990298 709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:10.990612 709 log.cc:824] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:10.991412 709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d: No bootstrap required, opened a new log
I20260808 10:03:10.993731 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:10.987550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.24.162.190:35199)
0808 10:03:10.987887 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:10.987898 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:10.987935 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:03:10.988499 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:10.988508 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:10.988589 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:10.988744 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:10.989000 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.990344 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.991497 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:10.992099 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:10.992234 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:10.992964 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:10.992968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:10.992968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:10.993176 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:10.993218 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:10.993232 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:10.993308 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:10.993379 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:03:10.993473 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":193,"threads_started":1}
I20260808 10:03:10.994037 709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } }
I20260808 10:03:10.994234 709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:10.994273 709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2857c8d54468435184838999a0d0f77d, State: Initialized, Role: FOLLOWER
I20260808 10:03:10.994407 709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } }
I20260808 10:03:10.994516 709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:10.994561 709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:10.994611 709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:10.994828 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:03:10.995285 709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } }
I20260808 10:03:10.995401 709 leader_election.cc:302] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2857c8d54468435184838999a0d0f77d; no voters:
I20260808 10:03:10.995558 709 leader_election.cc:288] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:10.995607 717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:10.995781 717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [term 1 LEADER]: Becoming Leader. State: Replica: 2857c8d54468435184838999a0d0f77d, State: Running, Role: LEADER
I20260808 10:03:10.995863 709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:10.995882 717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } }
I20260808 10:03:10.996399 722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2857c8d54468435184838999a0d0f77d. Latest consensus state: current_term: 1 leader_uuid: "2857c8d54468435184838999a0d0f77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } } }
I20260808 10:03:10.996399 719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2857c8d54468435184838999a0d0f77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2857c8d54468435184838999a0d0f77d" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35199 } } }
I20260808 10:03:10.996507 722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.996570 719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:10.996832 732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:10.997562 732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:10.999163 732 catalog_manager.cc:1347] Generated new cluster ID: 071a116bd148465dbb4b47606adcf7c3
I20260808 10:03:10.999228 732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:11.005749 732 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:11.006251 732 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:11.010169 732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2857c8d54468435184838999a0d0f77d: Generated new TSK 0
I20260808 10:03:11.010351 732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:11.838579 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:11.021058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.24.162.190:35199)
0808 10:03:11.021120 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:11.021122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:11.021226 (+ 104us) server_negotiation.cc:407] Connection header received
0808 10:03:11.021285 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:11.021286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:11.021299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:11.021388 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:03:11.021497 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:11.022307 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:11.023114 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:11.023445 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:11.023518 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:11.023645 (+ 127us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:03:11.838347 (+814702us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:03:11.838516 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51744: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 10:03:11.838680 714 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51744: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 10:03:11.838779 757 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:11.020975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35199 (local address 127.0.0.1:51744)
0808 10:03:11.021150 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:11.021161 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:11.021224 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:11.021384 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:11.021385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:11.021387 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:03:11.021441 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:11.021443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:11.022395 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:11.022396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:11.023037 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:11.023039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:11.023531 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:11.023531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:11.023570 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:11.023574 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:03:11.838499 (+814925us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:03:11.838628 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.162.190:35199: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260808 10:03:11.839334 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 607
2026-08-08T10:03:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1008 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:12 dist-test-slave-nfb5 krb5kdc[776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:12 dist-test-slave-nfb5 krb5kdc[776](info): set up 2 sockets
Aug 08 10:03:12 dist-test-slave-nfb5 krb5kdc[776](info): commencing operation
krb5kdc: starting...
W20260808 10:03:13.246238 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:13.367493 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40799
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:40799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:13.447548 799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:13.447772 799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:13.447794 799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:13.448967 799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:13.449012 799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:13.449026 799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:13.449066 799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:13.449096 799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:13.450542 799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:40799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:13.450795 799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:13.451046 799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:13.455790 806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.455865 808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.455883 805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:13.456003 799 server_base.cc:1034] running on GCE node
I20260808 10:03:13.456318 799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:13.456609 799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:13.457772 799 hybrid_clock.cc:648] HybridClock initialized: now 1786183393457757 us; error 48 us; skew 500 ppm
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:13.460250 799 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:13.460723 799 webserver.cc:460] Webserver started at http://127.24.162.190:35729/ using document root <none> and password file <none>
I20260808 10:03:13.460940 799 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:13.461004 799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:13.461151 799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:13.462168 799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "3d2424fd3a52417bb271b4f19ada3ec6"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "09d8600882f79fe628d92eedb51c2364"
server_key_iv: "c7e606ae3870e4c51564322ddc386648"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.462527 799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "3d2424fd3a52417bb271b4f19ada3ec6"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "09d8600882f79fe628d92eedb51c2364"
server_key_iv: "c7e606ae3870e4c51564322ddc386648"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.464107 799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:13.464910 817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.465123 799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:13.465227 799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "3d2424fd3a52417bb271b4f19ada3ec6"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "09d8600882f79fe628d92eedb51c2364"
server_key_iv: "c7e606ae3870e4c51564322ddc386648"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.465306 799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:13.493651 799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:13.495651 799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:13.495836 799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:13.500062 799 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:40799
I20260808 10:03:13.500106 899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:40799 every 8 connection(s)
I20260808 10:03:13.500491 799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:13.501394 900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:13.502040 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 799
I20260808 10:03:13.502142 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:13.502336 25226 external_mini_cluster.cc:1428] Setting key 23f24a22a8ddb5cc02f304c79f36094e
I20260808 10:03:13.504612 900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: Bootstrap starting.
I20260808 10:03:13.505237 900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:13.505528 900 log.cc:824] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:13.506321 900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: No bootstrap required, opened a new log
I20260808 10:03:13.508972 900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } }
I20260808 10:03:13.509107 900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:13.509130 900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d2424fd3a52417bb271b4f19ada3ec6, State: Initialized, Role: FOLLOWER
I20260808 10:03:13.509267 900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } }
I20260808 10:03:13.509331 900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:13.509353 900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:13.509425 900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:13.510018 900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } }
I20260808 10:03:13.510138 900 leader_election.cc:302] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d2424fd3a52417bb271b4f19ada3ec6; no voters:
I20260808 10:03:13.510317 900 leader_election.cc:288] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:13.510360 909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:13.510528 909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d2424fd3a52417bb271b4f19ada3ec6, State: Running, Role: LEADER
I20260808 10:03:13.510610 909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } }
I20260808 10:03:13.510614 900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:13.511066 911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } } }
I20260808 10:03:13.511053 909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d2424fd3a52417bb271b4f19ada3ec6. Latest consensus state: current_term: 1 leader_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d2424fd3a52417bb271b4f19ada3ec6" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40799 } } }
I20260808 10:03:13.511157 911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:13.511164 909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6 [sys.catalog]: This master's current role is: LEADER
W20260808 10:03:13.512974 936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:13.513037 936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:13.513094 920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:13.513243 920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:13.514245 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:13.502946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.24.162.190:40799)
0808 10:03:13.503128 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:13.503137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:13.503170 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:03:13.503574 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:13.503579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:13.503623 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:13.503748 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:13.504580 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.505620 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.506626 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.507089 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.507169 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:13.510049 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:13.510053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:13.510053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:13.510065 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:13.512587 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.513013 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.513016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.513017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.513126 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.513377 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.513378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.513378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.513602 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:13.513781 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:13.513877 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:03:13.514019 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 10:03:13.514937 920 catalog_manager.cc:1347] Generated new cluster ID: 93a9fb2f775b49249eee74373a1dcf01
I20260808 10:03:13.515017 920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:13.577045 920 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:13.577811 920 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:13.584990 920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d2424fd3a52417bb271b4f19ada3ec6: Generated new TSK 0
I20260808 10:03:13.585182 920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:13.614624 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:13.700906 943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:13.701109 943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:13.701133 943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:13.702263 943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:13.702306 943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:13.702354 943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:13.703895 943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:13.704095 943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:13.704317 943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:13.709252 957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.709246 950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.709275 949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:13.709335 943 server_base.cc:1034] running on GCE node
I20260808 10:03:13.709796 943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:13.710035 943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:13.711236 943 hybrid_clock.cc:648] HybridClock initialized: now 1786183393711213 us; error 36 us; skew 500 ppm
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:13.713886 943 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:13.714339 943 webserver.cc:460] Webserver started at http://127.24.162.129:40091/ using document root <none> and password file <none>
I20260808 10:03:13.714524 943 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:13.714564 943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:13.714653 943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:13.715708 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "fa0f3ff3cb214381a250693e20fe61e8"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "dce4f67d78492f30b89964564efa2db8"
server_key_iv: "f5bdb0ca110e33b2a632d560df78a962"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.716012 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "fa0f3ff3cb214381a250693e20fe61e8"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "dce4f67d78492f30b89964564efa2db8"
server_key_iv: "f5bdb0ca110e33b2a632d560df78a962"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.717337 943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:13.718022 965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.718194 943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:13.718276 943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "fa0f3ff3cb214381a250693e20fe61e8"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "dce4f67d78492f30b89964564efa2db8"
server_key_iv: "f5bdb0ca110e33b2a632d560df78a962"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.718331 943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:13.730192 943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:13.732535 943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:13.732650 943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:13.732844 943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:13.733104 943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:13.733135 943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.733157 943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:13.733170 943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.738682 943 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41395
I20260808 10:03:13.738807 1144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41395 every 8 connection(s)
I20260808 10:03:13.739109 943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:13.740052 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 943
I20260808 10:03:13.740166 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:13.740351 25226 external_mini_cluster.cc:1428] Setting key f6cedc575263051a92b34e7c64d00792
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:13.749436 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:13.739832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:35631 (local address 127.24.162.190:40799)
0808 10:03:13.739898 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:13.739900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:13.740171 (+ 271us) server_negotiation.cc:407] Connection header received
0808 10:03:13.741172 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:13.741173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:13.741193 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:13.741269 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:13.741836 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.742634 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.743667 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.744014 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.744085 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:13.746299 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:13.746300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:13.746300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:13.746307 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:13.747915 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.748424 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.748426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.748426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.748483 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.748777 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.748778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.748778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.748886 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:13.748938 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:13.749294 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 10:03:13.749370 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:03:13.749712 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:13.739835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40799 (local address 127.24.162.129:35631)
0808 10:03:13.740018 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:13.740055 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:13.741072 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:13.741329 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:13.741332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:13.741575 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:13.741752 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:13.741756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.742726 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.742727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:13.743576 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:13.743578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.744107 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.744108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:13.744201 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:13.744704 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:13.744712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:13.746203 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:13.748004 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:13.748012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:13.748014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:13.748343 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:13.748558 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:13.748559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:13.748560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:13.748707 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:13.748952 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:13.748960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:13.749136 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 10:03:13.749267 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 10:03:13.749457 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
I20260808 10:03:13.750397 1145 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40799
I20260808 10:03:13.750502 1145 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:13.750684 1145 heartbeater.cc:511] Master 127.24.162.190:40799 requested a full tablet report, sending...
I20260808 10:03:13.751299 844 ts_manager.cc:194] Registered new tserver with Master: fa0f3ff3cb214381a250693e20fe61e8 (127.24.162.129:41395)
I20260808 10:03:13.751875 844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:35631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:13.769074 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:13.850980 1154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:13.851222 1154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:13.851263 1154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:13.852429 1154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:13.852501 1154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:13.852552 1154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:13.854009 1154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:13.854215 1154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:13.854400 1154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:13.859396 1161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.859365 1163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:13.859400 1168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:13.859622 1154 server_base.cc:1034] running on GCE node
I20260808 10:03:13.859843 1154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:13.860031 1154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:13.861172 1154 hybrid_clock.cc:648] HybridClock initialized: now 1786183393861163 us; error 28 us; skew 500 ppm
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:13.863831 1154 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:13.864293 1154 webserver.cc:460] Webserver started at http://127.24.162.130:44253/ using document root <none> and password file <none>
I20260808 10:03:13.864514 1154 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:13.864578 1154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:13.864722 1154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:13.865898 1154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "30c18cc2fcf4484aa16df5532ab061e5"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "f856ad3ca58cfeb9b42cbd7e9fb97c69"
server_key_iv: "a96103afe85d78f25320ae731b702cee"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.866251 1154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "30c18cc2fcf4484aa16df5532ab061e5"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "f856ad3ca58cfeb9b42cbd7e9fb97c69"
server_key_iv: "a96103afe85d78f25320ae731b702cee"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.867689 1154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:13.868499 1177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.868666 1154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:13.868762 1154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "30c18cc2fcf4484aa16df5532ab061e5"
format_stamp: "Formatted at 2026-08-08 10:03:13 on dist-test-slave-nfb5"
server_key: "f856ad3ca58cfeb9b42cbd7e9fb97c69"
server_key_iv: "a96103afe85d78f25320ae731b702cee"
server_key_version: "encryptionkey@0"
I20260808 10:03:13.868848 1154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:13.884950 1154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:13.887274 1154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:13.887441 1154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:13.887667 1154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:13.887970 1154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:13.888023 1154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.888065 1154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:13.888091 1154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:13.894186 1154 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:36483
I20260808 10:03:13.894263 1364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:36483 every 8 connection(s)
I20260808 10:03:13.894593 1154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:13.904311 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 1154
I20260808 10:03:13.904414 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
Aug 08 10:03:13 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183393, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:13.904623 25226 external_mini_cluster.cc:1428] Setting key d27c87168fa6d4939e069754b5935643
I20260808 10:03:13.907811 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:13.895492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:60367 (local address 127.24.162.190:40799)
0808 10:03:13.895549 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:13.895551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:13.895659 (+ 108us) server_negotiation.cc:407] Connection header received
0808 10:03:13.897135 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:13.897137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:13.897150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:13.897224 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:13.897938 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.898760 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.900349 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.900713 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.900800 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:13.904917 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:13.904919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:13.904919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:13.904928 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:13.906391 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.906798 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.906799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.906800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.906861 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:13.907127 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:13.907128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:13.907129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:13.907242 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:13.907326 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:13.907640 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 10:03:13.907722 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:03:13.908030 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:13.895376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40799 (local address 127.24.162.130:60367)
0808 10:03:13.895531 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:13.895572 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:13.896990 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:13.897335 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:13.897339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:13.897587 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:13.897767 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:13.897771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.898884 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.898885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:13.900237 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:13.900240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:13.900812 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:13.900814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:13.900979 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:13.903001 (+ 2022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:13.903013 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:13.904774 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:13.906491 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:13.906498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:13.906500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:13.906721 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:13.906945 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:13.906946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:13.906946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:13.907058 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:13.907309 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:13.907316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:13.907456 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 10:03:13.907598 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 10:03:13.907760 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260808 10:03:13.908707 1365 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40799
I20260808 10:03:13.908810 1365 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:13.908999 1365 heartbeater.cc:511] Master 127.24.162.190:40799 requested a full tablet report, sending...
I20260808 10:03:13.909338 844 ts_manager.cc:194] Registered new tserver with Master: 30c18cc2fcf4484aa16df5532ab061e5 (127.24.162.130:36483)
I20260808 10:03:13.909770 844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:60367
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:13.937409 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:14.020967 1373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:14.021190 1373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:14.021216 1373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:14.022364 1373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:14.022403 1373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:14.022428 1373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:14.023866 1373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:14.024040 1373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:14.024204 1373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:14.029045 1373 server_base.cc:1034] running on GCE node
W20260808 10:03:14.028977 1379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:14.028982 1383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:14.029012 1378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:14.029451 1373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:14.029645 1373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:14.030778 1373 hybrid_clock.cc:648] HybridClock initialized: now 1786183394030750 us; error 46 us; skew 500 ppm
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.033294 1373 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:14.033672 1373 webserver.cc:460] Webserver started at http://127.24.162.131:43953/ using document root <none> and password file <none>
I20260808 10:03:14.033854 1373 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:14.033895 1373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:14.033984 1373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:14.034998 1373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "c2dfa4e05d484485b9b3d2e0663340c8"
format_stamp: "Formatted at 2026-08-08 10:03:14 on dist-test-slave-nfb5"
server_key: "a215d10efa03d0be5cf3b68427b52f94"
server_key_iv: "4fff06380ed43fdcb3edeec4d8d27af2"
server_key_version: "encryptionkey@0"
I20260808 10:03:14.035310 1373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "c2dfa4e05d484485b9b3d2e0663340c8"
format_stamp: "Formatted at 2026-08-08 10:03:14 on dist-test-slave-nfb5"
server_key: "a215d10efa03d0be5cf3b68427b52f94"
server_key_iv: "4fff06380ed43fdcb3edeec4d8d27af2"
server_key_version: "encryptionkey@0"
I20260808 10:03:14.036617 1373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:14.037297 1392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:14.037508 1373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:14.037623 1373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "c2dfa4e05d484485b9b3d2e0663340c8"
format_stamp: "Formatted at 2026-08-08 10:03:14 on dist-test-slave-nfb5"
server_key: "a215d10efa03d0be5cf3b68427b52f94"
server_key_iv: "4fff06380ed43fdcb3edeec4d8d27af2"
server_key_version: "encryptionkey@0"
I20260808 10:03:14.037710 1373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:14.067314 1373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:14.069744 1373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:14.069921 1373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:14.070155 1373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:14.070482 1373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:14.070536 1373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:14.070577 1373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:14.070604 1373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:14.076455 1373 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:37289
I20260808 10:03:14.076532 1562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:37289 every 8 connection(s)
I20260808 10:03:14.076853 1373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:14.083737 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 1373
I20260808 10:03:14.083827 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:14.084019 25226 external_mini_cluster.cc:1428] Setting key 883ffb24d029fa9476d99cae0d9f05be
I20260808 10:03:14.086555 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:14.077572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:39123 (local address 127.24.162.190:40799)
0808 10:03:14.077640 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.077642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.077854 (+ 212us) server_negotiation.cc:407] Connection header received
0808 10:03:14.078814 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.078816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.078829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.078888 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.079512 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.080023 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.081033 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.081249 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.081332 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:14.083569 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:14.083588 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:14.083588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:14.083604 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:14.085457 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.085807 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.085808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.085808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.085861 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.086069 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.086069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.086069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.086171 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:14.086220 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:14.086432 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:03:14.086491 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:03:14.086740 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:14.077549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40799 (local address 127.24.162.131:39123)
0808 10:03:14.077730 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:14.077766 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:14.078702 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:14.078990 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:14.078994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:14.079236 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:14.079419 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:14.079422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.080121 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.080123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:14.080938 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:14.080940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.081332 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.081333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:14.081456 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:14.081960 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:14.081969 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:14.083459 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:14.085525 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:14.085531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:14.085533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:14.085746 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:14.085917 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:14.085917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:14.085917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:14.086011 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:14.086219 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:14.086223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:14.086334 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:03:14.086416 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:03:14.086533 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260808 10:03:14.087244 1563 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40799
I20260808 10:03:14.087345 1563 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:14.087522 1563 heartbeater.cc:511] Master 127.24.162.190:40799 requested a full tablet report, sending...
I20260808 10:03:14.087862 844 ts_manager.cc:194] Registered new tserver with Master: c2dfa4e05d484485b9b3d2e0663340c8 (127.24.162.131:37289)
I20260808 10:03:14.088236 844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:39123
I20260808 10:03:14.095788 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/test-admin.keytab.
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.111284 25226 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 10:03:14.113039 1576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:14.112237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.24.162.129:41395)
0808 10:03:14.112425 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.112430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.112440 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:14.112465 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.112467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.112479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.112610 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.112931 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37192: BlockingRecv error: recv got EOF from 127.0.0.1:37192 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
W20260808 10:03:14.113135 25226 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.114101 25226 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.115046 25226 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:03:14.123016 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:14.115536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.24.162.129:41395)
0808 10:03:14.115594 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.115596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.115659 (+ 63us) server_negotiation.cc:407] Connection header received
0808 10:03:14.115740 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.115742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.115752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.115798 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.116209 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.116933 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.117654 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.117988 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.118069 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:14.120291 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:14.120321 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:14.120321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:14.120330 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:14.121836 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.122183 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.122191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.122191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.122243 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.122451 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.122452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.122452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.122584 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:14.122747 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:14.122807 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 10:03:14.122893 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 10:03:14.124809 1576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:14.124170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.24.162.129:41395)
0808 10:03:14.124225 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.124227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.124257 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:03:14.124327 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.124328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.124339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.124394 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.124763 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37208: BlockingRecv error: recv got EOF from 127.0.0.1:37208 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 10:03:14.124905 25226 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.125826 25226 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.126626 25226 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:03:14.132947 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:14.127196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.24.162.129:41395)
0808 10:03:14.127252 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.127254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.127266 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:14.127320 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.127320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.127328 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.127373 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.127680 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.128132 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.128749 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.128937 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.128991 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:14.130774 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:14.130775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:14.130776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:14.130782 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:14.131989 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.132295 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.132296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.132297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.132353 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.132516 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.132517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.132517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.132599 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:14.132668 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:14.132751 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 10:03:14.132811 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 10:03:14.134457 1576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:14.133840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.24.162.129:41395)
0808 10:03:14.133899 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.133902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.133962 (+ 60us) server_negotiation.cc:407] Connection header received
0808 10:03:14.134037 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.134038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.134047 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.134092 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.134405 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37222: BlockingRecv error: recv got EOF from 127.0.0.1:37222 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 10:03:14.134503 25226 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183366714887-25226-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.135486 25226 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:14.136346 25226 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:03:14.142577 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:14.136829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.24.162.129:41395)
0808 10:03:14.136882 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:14.136884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:14.136894 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:14.136944 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:14.136944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:14.136951 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:14.136995 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:14.137321 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.137735 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.138351 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:14.138521 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:14.138568 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:14.140494 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:14.140495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:14.140495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:14.140501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:14.141694 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.141987 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.141988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.141988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.142036 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:14.142181 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:14.142182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:14.142182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:14.142255 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:14.142314 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:14.142399 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:03:14.142453 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:03:14.143221 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 943
I20260808 10:03:14.161814 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 1154
I20260808 10:03:14.170998 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 1373
I20260808 10:03:14.177340 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 799
2026-08-08T10:03:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2160 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[1597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[1597](info): set up 2 sockets
Aug 08 10:03:14 dist-test-slave-nfb5 krb5kdc[1597](info): commencing operation
krb5kdc: starting...
W20260808 10:03:15.495194 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 10:03:15.716923 25226 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.222s user 0.001s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 10:03:15.920755 25226 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.204s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/test-user.keytab.
W20260808 10:03:16.807333 25226 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.791s user 0.001s sys 0.002s
Aug 08 10:03:16 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab.
Entry for principal oryx/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab.
I20260808 10:03:16.871887 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45989
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab
--principal=oryx/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:16.958168 1620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:16.958423 1620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:16.958461 1620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:16.959666 1620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:16.959728 1620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:16.959749 1620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:16.959772 1620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:16.959800 1620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:16.961333 1620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab
--principal=oryx/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:16.961621 1620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:16.961845 1620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:16.966663 1630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:16.966688 1627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:16.966817 1620 server_base.cc:1034] running on GCE node
W20260808 10:03:16.966701 1628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:16.967159 1620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:16.967454 1620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:16.968612 1620 hybrid_clock.cc:648] HybridClock initialized: now 1786183396968613 us; error 50 us; skew 500 ppm
Aug 08 10:03:16 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:16.974088 1620 init.cc:377] Logged in from keytab as oryx/127.24.162.190@KRBTEST.COM (short username oryx)
I20260808 10:03:16.974493 1620 webserver.cc:460] Webserver started at http://127.24.162.190:37715/ using document root <none> and password file <none>
I20260808 10:03:16.974674 1620 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:16.974712 1620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:16.974799 1620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:16.975880 1620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "9c2985209677472790c61be3594f09fe"
format_stamp: "Formatted at 2026-08-08 10:03:16 on dist-test-slave-nfb5"
server_key: "503bd44cf41f6790afbeee09184cb6b8"
server_key_iv: "11df2038d2f9b77e1618f52283677765"
server_key_version: "encryptionkey@0"
I20260808 10:03:16.976188 1620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "9c2985209677472790c61be3594f09fe"
format_stamp: "Formatted at 2026-08-08 10:03:16 on dist-test-slave-nfb5"
server_key: "503bd44cf41f6790afbeee09184cb6b8"
server_key_iv: "11df2038d2f9b77e1618f52283677765"
server_key_version: "encryptionkey@0"
I20260808 10:03:16.977571 1620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:16.978282 1637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:16.978466 1620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:16.978559 1620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "9c2985209677472790c61be3594f09fe"
format_stamp: "Formatted at 2026-08-08 10:03:16 on dist-test-slave-nfb5"
server_key: "503bd44cf41f6790afbeee09184cb6b8"
server_key_iv: "11df2038d2f9b77e1618f52283677765"
server_key_version: "encryptionkey@0"
I20260808 10:03:16.978618 1620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:16.992528 1620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:16.995083 1620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:16.995213 1620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:16.999605 1620 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:45989
I20260808 10:03:16.999639 1707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:45989 every 8 connection(s)
I20260808 10:03:17.000063 1620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:17.001142 1708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:17.003603 1708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Bootstrap starting.
I20260808 10:03:17.004215 1708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:17.004521 1708 log.cc:824] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:17.005343 1708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: No bootstrap required, opened a new log
I20260808 10:03:17.006984 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 1620
I20260808 10:03:17.007098 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:17.007134 1708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } }
I20260808 10:03:17.007295 25226 external_mini_cluster.cc:1428] Setting key 7a11fe66de354dba8594c42332669c92
I20260808 10:03:17.007330 1708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:17.007380 1708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2985209677472790c61be3594f09fe, State: Initialized, Role: FOLLOWER
I20260808 10:03:17.007530 1708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } }
I20260808 10:03:17.007627 1708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:17.007709 1708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:17.007774 1708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:17.008811 1708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } }
I20260808 10:03:17.008942 1708 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c2985209677472790c61be3594f09fe; no voters:
I20260808 10:03:17.009109 1708 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:17.009140 1713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:17.009370 1713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 LEADER]: Becoming Leader. State: Replica: 9c2985209677472790c61be3594f09fe, State: Running, Role: LEADER
I20260808 10:03:17.009385 1708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:17.009547 1713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } }
I20260808 10:03:17.009969 1714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } }
I20260808 10:03:17.010066 1714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:17.010269 1715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2985209677472790c61be3594f09fe. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } }
I20260808 10:03:17.010327 1715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:17.010860 1726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:17.011720 1726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:17.013432 1726 catalog_manager.cc:1347] Generated new cluster ID: fa7faa2828874bc6a82d97dd571e6ce3
I20260808 10:03:17.013497 1726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:17.018214 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.007863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49884 (local address 127.24.162.190:45989)
0808 10:03:17.008057 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.008067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.008103 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:03:17.008222 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.008227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.008271 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.008397 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.009307 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.010814 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.011749 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.012189 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.012329 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.014951 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.014955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.014956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.014968 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.016688 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.017072 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.017075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.017076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.017184 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.017402 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.017402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.017403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.017629 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.017805 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.017874 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.018018 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260808 10:03:17.051642 1726 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:17.052249 1726 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:17.055992 1726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Generated new TSK 0
I20260808 10:03:17.056201 1726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
I20260808 10:03:17.097745 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:17.208765 1743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:17.208985 1743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:17.209012 1743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:17.210520 1743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:17.210565 1743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:17.210597 1743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:17.212332 1743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:17.212602 1743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:17.212873 1743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.218457 1748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.218454 1751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.218470 1743 server_base.cc:1034] running on GCE node
W20260808 10:03:17.218454 1749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.218909 1743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.219173 1743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.220309 1743 hybrid_clock.cc:648] HybridClock initialized: now 1786183397220298 us; error 33 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.222882 1743 init.cc:377] Logged in from keytab as oryx/127.24.162.129@KRBTEST.COM (short username oryx)
I20260808 10:03:17.223443 1743 webserver.cc:460] Webserver started at http://127.24.162.129:33975/ using document root <none> and password file <none>
I20260808 10:03:17.223694 1743 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.223742 1743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.223886 1743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:17.224920 1743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "e6745ab226ca4db683d23cb5782e8e21"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "292586404d924e74a892e51efe42287e"
server_key_iv: "fa379e46e0a4a91472e582c4c5e938c6"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.225284 1743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "e6745ab226ca4db683d23cb5782e8e21"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "292586404d924e74a892e51efe42287e"
server_key_iv: "fa379e46e0a4a91472e582c4c5e938c6"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.226627 1743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:17.227516 1760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.227716 1743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:17.227815 1743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "e6745ab226ca4db683d23cb5782e8e21"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "292586404d924e74a892e51efe42287e"
server_key_iv: "fa379e46e0a4a91472e582c4c5e938c6"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.227895 1743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.240597 1743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:17.243063 1743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:17.243232 1743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:17.243480 1743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:17.243805 1743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:17.243860 1743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.243904 1743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:17.243932 1743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.249996 1743 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:46871
I20260808 10:03:17.250041 1923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:46871 every 8 connection(s)
I20260808 10:03:17.250370 1743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:17.254213 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 1743
I20260808 10:03:17.254341 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:17.254556 25226 external_mini_cluster.cc:1428] Setting key 030fac6a67b8645e82b8cf34d4680254
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:17.260759 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.251023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:42441)
0808 10:03:17.251210 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.251248 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.251763 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.252032 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.252036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.252609 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.252805 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.252809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.253535 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.253537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.254457 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.254461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.255017 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.255019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.255153 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.255740 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.255747 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.257384 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.259284 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.259291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.259293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.259538 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.259767 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.259768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.259768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.259928 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.260249 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.260255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.260371 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.260455 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.260582 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260808 10:03:17.260762 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.251018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:42441 (local address 127.24.162.190:45989)
0808 10:03:17.251098 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.251101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.251340 (+ 239us) server_negotiation.cc:407] Connection header received
0808 10:03:17.251872 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.251874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.251891 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.251954 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.252912 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.253436 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.254560 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.254905 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.255008 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.257478 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.257480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.257480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.257489 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.259185 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.259619 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.259621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.259621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.259699 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.260010 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.260011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.260011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.260176 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.260246 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.260506 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.260612 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:03:17.261649 1924 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:17.261762 1924 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:17.261977 1924 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:17.262522 1659 ts_manager.cc:194] Registered new tserver with Master: e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871)
I20260808 10:03:17.263190 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:42441
WARNING: no policy specified for oryx/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
I20260808 10:03:17.282693 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:17.366961 1932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:17.367154 1932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:17.367177 1932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:17.368238 1932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:17.368274 1932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:17.368299 1932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:17.369643 1932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:17.369834 1932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:17.370010 1932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.375190 1938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.375202 1945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.375267 1932 server_base.cc:1034] running on GCE node
W20260808 10:03:17.375269 1939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.375702 1932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.375948 1932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.377105 1932 hybrid_clock.cc:648] HybridClock initialized: now 1786183397377088 us; error 30 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.379710 1932 init.cc:377] Logged in from keytab as oryx/127.24.162.130@KRBTEST.COM (short username oryx)
I20260808 10:03:17.380131 1932 webserver.cc:460] Webserver started at http://127.24.162.130:36669/ using document root <none> and password file <none>
I20260808 10:03:17.380385 1932 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.380450 1932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.380601 1932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:17.381719 1932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "2e421faeab2148f4b0cddfd67a561be2"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "c7f40e3f1c52c4917abdf205111f410c"
server_key_iv: "0ca7c57d25bb243398175825b8654b3b"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.382123 1932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "2e421faeab2148f4b0cddfd67a561be2"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "c7f40e3f1c52c4917abdf205111f410c"
server_key_iv: "0ca7c57d25bb243398175825b8654b3b"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.383533 1932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:17.384235 1956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.384414 1932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:17.384516 1932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "2e421faeab2148f4b0cddfd67a561be2"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "c7f40e3f1c52c4917abdf205111f410c"
server_key_iv: "0ca7c57d25bb243398175825b8654b3b"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.384588 1932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.396143 1932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:17.398005 1932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:17.398118 1932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:17.398311 1932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:17.398556 1932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:17.398586 1932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.398608 1932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:17.398626 1932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.404558 1932 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40741
I20260808 10:03:17.404637 2121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40741 every 8 connection(s)
I20260808 10:03:17.405033 1932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:17.407956 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 1932
I20260808 10:03:17.408038 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:17.408234 25226 external_mini_cluster.cc:1428] Setting key edde24153678eebb5097d82f3b356b26
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:17.415561 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.405773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:36181 (local address 127.24.162.190:45989)
0808 10:03:17.405850 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.405853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.406045 (+ 192us) server_negotiation.cc:407] Connection header received
0808 10:03:17.406594 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.406596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.406610 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.406675 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.407637 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.408241 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.409515 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.409761 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.409871 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.412379 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.412381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.412381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.412389 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.414055 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.414461 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.414463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.414463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.414531 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.414779 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.414780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.414780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.414905 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.415066 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.415377 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.415466 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:03:17.415772 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.405755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:36181)
0808 10:03:17.405919 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.405960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.406486 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.406745 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.406749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.407347 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.407532 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.407537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.408385 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.408387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.409428 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.409431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.409887 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.409888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.410034 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.410535 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.410545 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.412276 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.414136 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.414142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.414144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.414360 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.414607 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.414607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.414608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.414714 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.415054 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.415061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.415241 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.415353 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.415515 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260808 10:03:17.416307 2123 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:17.416432 2123 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:17.416638 2123 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:17.417011 1659 ts_manager.cc:194] Registered new tserver with Master: 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741)
I20260808 10:03:17.417388 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:36181
WARNING: no policy specified for oryx/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
I20260808 10:03:17.437263 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:17.521839 2134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:17.522071 2134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:17.522113 2134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:17.523336 2134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:17.523396 2134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:17.523439 2134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:17.525425 2134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:45989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:17.525703 2134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:17.525941 2134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.531023 2144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.531061 2143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.531132 2134 server_base.cc:1034] running on GCE node
W20260808 10:03:17.531061 2146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.531484 2134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.531734 2134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.532898 2134 hybrid_clock.cc:648] HybridClock initialized: now 1786183397532893 us; error 41 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.535460 2134 init.cc:377] Logged in from keytab as oryx/127.24.162.131@KRBTEST.COM (short username oryx)
I20260808 10:03:17.535940 2134 webserver.cc:460] Webserver started at http://127.24.162.131:42435/ using document root <none> and password file <none>
I20260808 10:03:17.536194 2134 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.536260 2134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.536392 2134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:17.537421 2134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "a57f521fabc64852a18e8b6b18bda148"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "520d59a95e87971247b45eb85958b56e"
server_key_iv: "02ff2bc711604575fce73d94d4a49aa8"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.537797 2134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "a57f521fabc64852a18e8b6b18bda148"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "520d59a95e87971247b45eb85958b56e"
server_key_iv: "02ff2bc711604575fce73d94d4a49aa8"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.539193 2134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:03:17.539968 2157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.540149 2134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:17.540249 2134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "a57f521fabc64852a18e8b6b18bda148"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "520d59a95e87971247b45eb85958b56e"
server_key_iv: "02ff2bc711604575fce73d94d4a49aa8"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.540323 2134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.547505 2134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:17.549221 2134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:17.549379 2134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:17.549592 2134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:17.549881 2134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:17.549932 2134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.549973 2134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:17.550009 2134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.556056 2134 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:44293
I20260808 10:03:17.556134 2328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:44293 every 8 connection(s)
I20260808 10:03:17.556483 2134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:17.562551 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 2134
I20260808 10:03:17.562645 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:17.562832 25226 external_mini_cluster.cc:1428] Setting key 7827738374adbd386d9e749273729f44
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:17.567703 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.557235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:36031 (local address 127.24.162.190:45989)
0808 10:03:17.557303 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.557305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.557500 (+ 195us) server_negotiation.cc:407] Connection header received
0808 10:03:17.558010 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.558011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.558020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.558063 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.559024 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.559528 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.560555 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.560802 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.560871 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.563860 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.563862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.563862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.563871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.565866 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.566467 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.566468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.566468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.566537 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.566867 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.566868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.566868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.567060 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.567135 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.567503 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.567594 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:03:17.567912 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.557214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:36031)
0808 10:03:17.557372 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.557414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.557908 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.558114 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.558118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.558700 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.558888 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.558892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.559627 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.559629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.560475 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.560478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.560899 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.560900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.561072 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.561670 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.561680 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.563775 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.565951 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.565958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.565960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.566362 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.566602 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.566603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.566603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.566776 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.567150 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.567155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.567334 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.567456 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.567653 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260808 10:03:17.568506 2329 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:17.568612 2329 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:17.568805 2329 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:17.569248 1659 ts_manager.cc:194] Registered new tserver with Master: a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:17.569711 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:36031
I20260808 10:03:17.574781 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.162.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.189.keytab.
Entry for principal oryx/127.24.162.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.189.keytab.
I20260808 10:03:17.600066 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:45893
--webserver_interface=127.24.162.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.189.keytab
--principal=oryx/127.24.162.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45989,127.24.162.189:45893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:17.683908 2337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:17.684103 2337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:17.684125 2337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:17.685266 2337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:17.685307 2337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:17.685322 2337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:17.685335 2337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:17.685348 2337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:17.686723 2337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45989,127.24.162.189:45893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.189.keytab
--principal=oryx/127.24.162.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:45893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.162.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
W20260808 10:03:17.686893 2337 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.162.190:45989,127.24.162.189:45893'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 10:03:17.687057 2337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:17.687237 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.692047 2342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.692111 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.692055 2345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.692162 2337 server_base.cc:1034] running on GCE node
I20260808 10:03:17.692422 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.692718 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.693879 2337 hybrid_clock.cc:648] HybridClock initialized: now 1786183397693869 us; error 42 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.696362 2337 init.cc:377] Logged in from keytab as oryx/127.24.162.189@KRBTEST.COM (short username oryx)
I20260808 10:03:17.696846 2337 webserver.cc:460] Webserver started at http://127.24.162.189:43383/ using document root <none> and password file <none>
I20260808 10:03:17.697081 2337 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.697149 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.697295 2337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:17.698307 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/instance:
uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "32526b45d408012d8f0364166c3fecaf"
server_key_iv: "929c624c39f6bd7fbf52c9ba16c0e559"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.698660 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal/instance:
uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "32526b45d408012d8f0364166c3fecaf"
server_key_iv: "929c624c39f6bd7fbf52c9ba16c0e559"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.700055 2337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:17.700806 2354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.700974 2337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:17.701061 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "32526b45d408012d8f0364166c3fecaf"
server_key_iv: "929c624c39f6bd7fbf52c9ba16c0e559"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.701141 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.726542 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:17.729125 2337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:17.729290 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:17.733678 2337 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.189:45893
I20260808 10:03:17.733748 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.189:45893 every 8 connection(s)
I20260808 10:03:17.734114 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
I20260808 10:03:17.735162 2431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:17.735280 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 2337
I20260808 10:03:17.737627 2431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.189@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:17.747555 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.738395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.24.162.190:45989)
0808 10:03:17.738463 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.738465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.738632 (+ 167us) server_negotiation.cc:407] Connection header received
0808 10:03:17.738697 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.738697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.738707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.738759 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.739321 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.740036 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.741432 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.741732 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.741816 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.744449 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.744450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.744450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.744459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.745894 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.746395 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.746396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.746396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.746485 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.746839 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.746839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.746840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.747059 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.747142 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.747370 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.747461 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:03:17.747779 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.736151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.24.162.189:45893)
0808 10:03:17.736347 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.736357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.736390 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:03:17.736890 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.736899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.736981 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.737138 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.737725 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.739019 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.740104 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.740801 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.740908 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.743871 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.743879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.743879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.743885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.745731 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.746411 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.746414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.746415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.746525 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.746763 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.746763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.746764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.747106 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.747243 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.747361 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.747486 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260808 10:03:17.747782 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.738354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35814)
0808 10:03:17.738526 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.738565 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.738637 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.738795 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.738798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.739182 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.739249 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.739251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.740137 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.740138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.741045 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.741047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.741820 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.741821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.741872 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.742358 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.742366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.744368 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.745974 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.745980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.745982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.746318 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.746601 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.746602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.746602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.746770 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.747131 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.747136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.747226 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.747338 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.747566 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":65,"thread_start_us":99,"threads_started":1}
I20260808 10:03:17.748471 2431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.189@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
I20260808 10:03:17.755234 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.748728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47002)
0808 10:03:17.748780 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.748791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.748839 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.749023 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.749023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.749162 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.749189 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.749190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.749797 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.749798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.750269 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.750270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.750650 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.750651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.750712 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.751144 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.751149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.752299 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.754191 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.754192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.754193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.754356 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.754542 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.754543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.754543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.754596 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.754839 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.754839 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.754875 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.754973 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.755074 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 10:03:17.755275 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.748781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.24.162.189:45893)
0808 10:03:17.748810 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.748812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.748831 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:03:17.748962 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.748963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.748973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.749021 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.749259 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.749721 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.750360 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.750567 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.750650 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.752404 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.752405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.752405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.752443 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.754069 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.754424 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.754425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.754425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.754479 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.754662 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.754663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.754663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.754776 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.754848 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.754996 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.755124 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":15}
I20260808 10:03:17.756723 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Bootstrap starting.
I20260808 10:03:17.757543 2431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:17.757778 2431 log.cc:824] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:17.758481 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: No bootstrap required, opened a new log
I20260808 10:03:17.759917 2431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } }
I20260808 10:03:17.760039 2431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:17.760062 2431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006fee5cbbfa4121a883bf96bcf1f1ff, State: Initialized, Role: FOLLOWER
I20260808 10:03:17.760190 2431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } }
I20260808 10:03:17.760442 2431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:17.760424 2437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } } }
I20260808 10:03:17.760543 2437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:03:17.762079 2453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:17.762140 2453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:17.763918 2337 master_runner.cc:428] Detected that this master 006fee5cbbfa4121a883bf96bcf1f1ff is joining an existing cluster
I20260808 10:03:17.763984 2337 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.162.189:45893
I20260808 10:03:17.771768 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.765319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47012)
0808 10:03:17.765451 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.765460 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.765517 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.765645 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.765645 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.765841 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.765881 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.765882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.766474 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.766475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.767390 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.767393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.767809 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.767844 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.768325 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.768329 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.768915 (+ 586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.770678 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.770679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.770680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.770847 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.771077 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.771079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.771133 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.771386 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.771386 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.771423 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.771512 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.771626 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260808 10:03:17.772189 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.765440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.24.162.189:45893)
0808 10:03:17.765479 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.765480 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.765506 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:03:17.765575 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.765576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.765585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.765642 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.765957 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.766397 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767478 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.767711 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767748 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.769015 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.769016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.769016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.769022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.770585 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.770959 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.770960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.770960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.771017 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771215 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.771215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.771216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.771325 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.771394 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.771544 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.771702 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":22}
I20260808 10:03:17.772342 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.765139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.24.162.190:45989)
0808 10:03:17.765184 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.765187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.765196 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:17.765280 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.765281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.765294 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.765371 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.765768 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.766561 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767564 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.767868 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.769005 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.769006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.769006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.769014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.771038 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771471 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.771473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.771473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.771534 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771753 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.771754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.771754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.771962 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.771997 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.772171 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.772254 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:03:17.772554 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.764965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35828)
0808 10:03:17.765117 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.765130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.765193 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.765375 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.765376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.765596 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.765689 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.765691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.766650 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.766651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.767489 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.767490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.767953 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.767954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.768016 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.768487 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.768491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.768915 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.771133 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.771135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.771370 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.771609 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.771609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.771610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.771669 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.772034 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.772035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.772064 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.772132 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.772233 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.190@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
I20260808 10:03:17.781850 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.774594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.24.162.189:45893)
0808 10:03:17.774632 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.774634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.774793 (+ 159us) server_negotiation.cc:407] Connection header received
0808 10:03:17.774856 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.774857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.774866 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.774955 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.775345 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.776060 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.776844 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.777139 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.777211 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.779203 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.779204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.779205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.779212 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.780698 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.781044 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.781045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.781045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.781104 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.781323 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.781323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.781324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.781448 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.781519 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.781688 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.781762 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:17.781884 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.774548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47028)
0808 10:03:17.774684 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.774729 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.774800 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.774959 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.774962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.775213 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.775268 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.775270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.776149 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.776150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.776761 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.776764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.777234 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.777235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.777293 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.777736 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.777741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.779126 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.780770 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.780777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.780779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.780973 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.781179 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.781180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.781180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.781257 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.781519 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.781526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.781579 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.781647 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.781748 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
I20260808 10:03:17.782799 1659 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.162.189:45893: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
member_type: NON_VOTER
last_known_addr {
host: "127.24.162.189"
port: 45893
}
attrs {
promote: true
}
}
dest_uuid: "9c2985209677472790c61be3594f09fe"
cas_config_opid_index: -1
I20260808 10:03:17.783214 1659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } }
I20260808 10:03:17.783768 2465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } }
I20260808 10:03:17.784130 2393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:17.784965 2465 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.162.189:45893
I20260808 10:03:17.785087 2393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 10:03:17.785144 2465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } }
I20260808 10:03:17.785218 2465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:17.785292 2337 master.cc:537] Master@127.24.162.189:45893 shutting down...
I20260808 10:03:17.787639 2465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:17.788379 2337 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:17.788475 2337 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:17.788533 2337 tablet_replica.cc:330] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: stopping tablet replica
W20260808 10:03:17.788537 1643 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 10:03:17.788966 1643 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe -> Peer 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189:45893): Couldn't send request to peer 006fee5cbbfa4121a883bf96bcf1f1ff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:03:17.801808 2337 master.cc:552] Master@127.24.162.189:45893 shutdown complete.
I20260808 10:03:17.801868 2337 master_runner.cc:305] Clearing existing system tablet
I20260808 10:03:17.802007 2337 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.802058 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.802798 2337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:03:17.803375 2474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.803520 2337 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.803584 2337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.803609 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "32526b45d408012d8f0364166c3fecaf"
server_key_iv: "929c624c39f6bd7fbf52c9ba16c0e559"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.804488 2337 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:17.806180 2337 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:03:17.806258 2337 log.cc:1196] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 10:03:17.806481 2337 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Deleting consensus metadata
I20260808 10:03:17.806607 2337 master_runner.cc:315] Copying system tablet from 127.24.162.190:45989
I20260808 10:03:17.807278 2337 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: tablet copy: Beginning tablet copy session from remote peer at address 127.24.162.190:45989
I20260808 10:03:17.813957 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.807608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35842)
0808 10:03:17.807773 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:17.807783 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:17.807849 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:17.807992 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:17.807993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:17.808145 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:17.808207 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.808209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.809143 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.809144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.809865 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:17.809868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.810323 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.810324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:17.810372 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.810949 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:17.810956 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:17.811263 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:17.813000 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.813001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.813001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.813170 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.813392 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:17.813393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:17.813393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:17.813446 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:17.813704 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:17.813705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:17.813745 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:03:17.813809 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:03:17.813885 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
I20260808 10:03:17.814030 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.807687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.24.162.190:45989)
0808 10:03:17.807742 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.807744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.807844 (+ 100us) server_negotiation.cc:407] Connection header received
0808 10:03:17.807916 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.807917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.807929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.807990 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.808268 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.809054 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.809939 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.810251 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.810334 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.811349 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.811351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.811352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.811359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.812915 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.813255 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.813256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.813257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.813328 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.813501 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.813502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.813502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.813634 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.813699 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.813850 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.813933 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:03:17.814313 1688 tablet_copy_service.cc:140] P 9c2985209677472790c61be3594f09fe: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 006fee5cbbfa4121a883bf96bcf1f1ff ({username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:35842)
I20260808 10:03:17.814466 1688 tablet_copy_service.cc:161] P 9c2985209677472790c61be3594f09fe: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 006fee5cbbfa4121a883bf96bcf1f1ff at {username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:35842: session id = 006fee5cbbfa4121a883bf96bcf1f1ff-00000000000000000000000000000000
I20260808 10:03:17.815467 1688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:03:17.816111 2337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:03:17.817462 2337 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: tablet copy: Starting download of 0 data blocks...
I20260808 10:03:17.817617 2337 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: tablet copy: Starting download of 1 WAL segments...
I20260808 10:03:17.818536 2337 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:03:17.819459 1688 tablet_copy_service.cc:342] P 9c2985209677472790c61be3594f09fe: Request end of tablet copy session 006fee5cbbfa4121a883bf96bcf1f1ff-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:35842
I20260808 10:03:17.819522 1688 tablet_copy_service.cc:433] P 9c2985209677472790c61be3594f09fe: ending tablet copy session 006fee5cbbfa4121a883bf96bcf1f1ff-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 006fee5cbbfa4121a883bf96bcf1f1ff
I20260808 10:03:17.822355 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.823451 2486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.823618 2488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.823887 2491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.824034 2337 server_base.cc:1034] running on GCE node
I20260808 10:03:17.824136 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.824265 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.825393 2337 hybrid_clock.cc:648] HybridClock initialized: now 1786183397825387 us; error 37 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.826751 2337 init.cc:377] Logged in from keytab as oryx/127.24.162.189@KRBTEST.COM (short username oryx)
I20260808 10:03:17.827061 2337 webserver.cc:460] Webserver started at http://127.24.162.189:43559/ using document root <none> and password file <none>
I20260808 10:03:17.827186 2337 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.827239 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.827840 2337 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:17.828387 2501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:17.828526 2337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:17.828580 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "32526b45d408012d8f0364166c3fecaf"
server_key_iv: "929c624c39f6bd7fbf52c9ba16c0e559"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.828650 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:17.836651 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:17.838176 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:17.841254 2337 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.189:45893
I20260808 10:03:17.841518 2573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.189:45893 every 8 connection(s)
I20260808 10:03:17.841648 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
I20260808 10:03:17.842546 2574 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:03:17.843104 2574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Bootstrap starting.
I20260808 10:03:17.846338 2574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:03:17.846590 2574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Bootstrap complete.
I20260808 10:03:17.846804 2574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } }
I20260808 10:03:17.846894 2574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 006fee5cbbfa4121a883bf96bcf1f1ff, State: Initialized, Role: LEARNER
I20260808 10:03:17.846958 2574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } }
I20260808 10:03:17.847155 2574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:17.847170 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } }
I20260808 10:03:17.847321 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: LEARNER
I20260808 10:03:17.848769 2598 catalog_manager.cc:1259] Loaded cluster ID: fa7faa2828874bc6a82d97dd571e6ce3
I20260808 10:03:17.848814 2598 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: loading cluster ID for follower catalog manager: success
I20260808 10:03:17.849910 2598 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: acquiring CA information for follower catalog manager: success
I20260808 10:03:17.850234 2598 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:03:17.856866 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:17.849557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.24.162.189:45893)
0808 10:03:17.849657 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:17.849659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:17.849703 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:03:17.849773 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:17.849773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:17.849913 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:17.850687 (+ 774us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:17.850730 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.851415 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.852452 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:17.852638 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:17.852692 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:17.854178 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:17.854179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:17.854179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:17.854199 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:17.855663 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.856094 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.856095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.856096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.856166 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:17.856402 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:17.856402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:17.856403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:17.856500 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:17.856577 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:17.856735 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:03:17.856803 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":53,"threads_started":1}
WARNING: no policy specified for oryx/127.24.162.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.188.keytab.
Entry for principal oryx/127.24.162.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.188.keytab.
I20260808 10:03:17.885574 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:33055
--webserver_interface=127.24.162.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.188.keytab
--principal=oryx/127.24.162.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:17.982592 2606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:17.982846 2606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:17.982872 2606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:17.984513 2606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:17.984556 2606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:17.984571 2606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:17.984584 2606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:17.984612 2606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:17.986125 2606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.188.keytab
--principal=oryx/127.24.162.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.162.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:17.986390 2606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:17.986704 2606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:17.991775 2615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.991801 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:17.991839 2613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:17.992005 2606 server_base.cc:1034] running on GCE node
I20260808 10:03:17.992259 2606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:17.992542 2606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:17.993763 2606 hybrid_clock.cc:648] HybridClock initialized: now 1786183397993755 us; error 36 us; skew 500 ppm
Aug 08 10:03:17 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:17.996343 2606 init.cc:377] Logged in from keytab as oryx/127.24.162.188@KRBTEST.COM (short username oryx)
I20260808 10:03:17.996814 2606 webserver.cc:460] Webserver started at http://127.24.162.188:42645/ using document root <none> and password file <none>
I20260808 10:03:17.997030 2606 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:17.997076 2606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:17.997233 2606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:17.998260 2606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/instance:
uuid: "2b079526c52a49deb541ea4fae2a929f"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "88363b3b71617cbf2a3aa1f7eb86eee6"
server_key_iv: "076329299ee72cbd4128e217bd1b2a72"
server_key_version: "encryptionkey@0"
I20260808 10:03:17.998620 2606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal/instance:
uuid: "2b079526c52a49deb541ea4fae2a929f"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "88363b3b71617cbf2a3aa1f7eb86eee6"
server_key_iv: "076329299ee72cbd4128e217bd1b2a72"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.000064 2606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:18.000972 2625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.001166 2606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:18.001266 2606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "2b079526c52a49deb541ea4fae2a929f"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "88363b3b71617cbf2a3aa1f7eb86eee6"
server_key_iv: "076329299ee72cbd4128e217bd1b2a72"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.001354 2606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:18.029480 2606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:18.032027 2606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:18.032200 2606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:18.037046 2606 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.188:33055
I20260808 10:03:18.037094 2699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.188:33055 every 8 connection(s)
I20260808 10:03:18.037474 2606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
I20260808 10:03:18.038458 2700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:18.040125 2700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:03:18.041646 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 2606
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.188@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.050437 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.040974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.24.162.190:45989)
0808 10:03:18.041043 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.041046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.041310 (+ 264us) server_negotiation.cc:407] Connection header received
0808 10:03:18.042034 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.042035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.042048 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.042105 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.042877 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.043522 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.044572 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.044763 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.044816 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.047440 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.047442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.047442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.047450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.049067 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.049477 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.049478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.049479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.049537 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.049799 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.049800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.049801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.049916 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.049991 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.050259 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.050344 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:03:18.050628 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.040861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35850)
0808 10:03:18.041071 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.041115 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.041931 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.042189 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.042193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.042535 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.042786 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.042792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.043708 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.043709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.044487 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.044491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.044865 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.044866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.045004 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.045543 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.045551 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.047353 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.049163 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.049169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.049172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.049389 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.049613 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.049613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.049614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.049734 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.049982 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.049989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.050123 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.050222 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.050370 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260808 10:03:18.051167 2700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:03:18.051486 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.042263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.24.162.188:33055)
0808 10:03:18.042436 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.042439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.042455 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:03:18.042473 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.042475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.042493 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.042653 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.043347 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.044411 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.045345 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.045629 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.045704 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.048488 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.048500 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.048500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.048506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.050071 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.050443 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.050446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.050447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.050542 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.050780 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.050781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.050781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.051041 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.051155 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.051319 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.051397 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.188@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
I20260808 10:03:18.057706 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.051436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47056)
0808 10:03:18.051489 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.051500 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.051771 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.051932 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.051933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.052070 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.052121 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.052122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.052785 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.052785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.053266 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.053267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.053577 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.053578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.053626 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.053949 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.053954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.055158 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.056819 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.056820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.056820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.056996 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.057192 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.057193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.057193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.057267 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.057480 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.057480 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.057516 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.057568 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.057636 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:03:18.057715 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.051500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.24.162.189:45893)
0808 10:03:18.051544 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.051546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.051555 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:18.051849 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.051850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.051865 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.051928 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.052208 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.052714 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.053334 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.053519 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.053568 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.055241 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.055242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.055242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.055250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.056712 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.057063 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.057064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.057064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.057114 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.057326 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.057327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.057327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.057424 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.057475 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.057585 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.057646 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:18.058131 2700 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183397, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.188@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.063967 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.058388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:55992)
0808 10:03:18.058433 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.058443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.058483 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.058586 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.058587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.058694 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.058718 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.058719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.059359 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.059360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.059808 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.059810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.060304 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.060305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.060343 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.060708 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.060712 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.061769 (+ 1057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.063169 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.063170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.063171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.063328 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.063491 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.063492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.063492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.063541 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.063747 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.063747 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.063779 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.063841 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.063912 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:03:18.064028 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.058414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.24.162.188:33055)
0808 10:03:18.058462 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.058464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.058474 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:18.058523 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.058523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.058531 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.058574 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.058778 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.059291 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.060032 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.060236 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.060296 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.061862 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.061863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.061863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.061869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.063093 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.063386 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.063387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.063387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.063435 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.063587 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.063587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.063588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.063673 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.063746 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.063859 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.063956 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":28}
I20260808 10:03:18.065291 2700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Bootstrap starting.
I20260808 10:03:18.066126 2700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:18.066421 2700 log.cc:824] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:18.067204 2700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: No bootstrap required, opened a new log
I20260808 10:03:18.068746 2700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } }
I20260808 10:03:18.068900 2700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:18.068948 2700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b079526c52a49deb541ea4fae2a929f, State: Initialized, Role: FOLLOWER
I20260808 10:03:18.069099 2700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } }
I20260808 10:03:18.069381 2700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:18.069358 2708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } } }
I20260808 10:03:18.069468 2708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:03:18.071128 2722 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:18.071211 2722 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:18.073378 2606 master_runner.cc:428] Detected that this master 2b079526c52a49deb541ea4fae2a929f is joining an existing cluster
I20260808 10:03:18.073439 2606 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.162.188:33055
I20260808 10:03:18.081570 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.074585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.24.162.190:45989)
0808 10:03:18.074635 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.074637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.074787 (+ 150us) server_negotiation.cc:407] Connection header received
0808 10:03:18.074859 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.074860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.074870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.074938 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.075427 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076180 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.077291 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.077604 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.077665 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.078809 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.078810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.078811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.078817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.080361 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.080776 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.080777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.080777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.080831 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.081070 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.081071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.081071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.081169 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.081232 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.081443 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.081506 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:03:18.081650 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.074553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35862)
0808 10:03:18.074713 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.074722 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.074800 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.074944 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.074944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.075255 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.075319 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.075321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076298 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.076299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.077176 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.077179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.077736 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.077737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.077797 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.078264 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.078269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.078720 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.080443 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.080444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.080444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.080693 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.080916 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.080917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.080917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.081008 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.081249 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.081251 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.081315 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.081407 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.081525 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183396, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.190@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.103353 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 1743
I20260808 10:03:18.103672 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.074760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47060)
0808 10:03:18.075013 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.075024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.075209 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.075349 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.075349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.075540 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.075607 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.075608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076728 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.076729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.077652 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.077654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.078145 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.078145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.078234 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.099509 (+ 21275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.099514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.100103 (+ 589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.102085 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.102087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.102510 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.102830 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.102832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.102895 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.103246 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.103246 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.103294 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.103375 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.103497 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":38,"thread_start_us":75,"threads_started":1}
I20260808 10:03:18.103820 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.074984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.24.162.188:33055)
0808 10:03:18.075024 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.075026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.075508 (+ 482us) server_negotiation.cc:407] Connection header received
0808 10:03:18.075598 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.075598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.075606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.075631 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.076016 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076644 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.077750 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.078024 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.078061 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.100465 (+ 22404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.100466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.100467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.100473 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.102614 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.103009 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.103010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.103011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.103060 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.103237 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.103237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.103238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.103350 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.103397 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.103676 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.103748 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":21}
I20260808 10:03:18.104108 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.075029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56008)
0808 10:03:18.075459 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.075468 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.075536 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.075702 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.075702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.075879 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.075940 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.075941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076731 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.076731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.077652 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.077654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.078127 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.078128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.078237 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.099805 (+ 21568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.099810 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.100361 (+ 551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.102680 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.102682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.102892 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.103122 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.103123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.103123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.103168 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.103422 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.103423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.103526 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.103917 (+ 391us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.103999 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":34,"thread_start_us":63,"threads_started":1}
I20260808 10:03:18.124050 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.074657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.24.162.189:45893)
0808 10:03:18.074695 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.074697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.075213 (+ 516us) server_negotiation.cc:407] Connection header received
0808 10:03:18.075263 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.075264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.075275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.075343 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.075698 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.076628 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.077751 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.078042 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.078079 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.100249 (+ 22170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.100251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.100251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.100260 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.101983 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102700 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.102701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.102701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.102757 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102995 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.102996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.102996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.103140 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.103215 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.123783 (+ 20568us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.123929 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:03:18.126148 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.083839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.24.162.188:33055)
0808 10:03:18.084012 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.084014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.084019 (+ 5us) server_negotiation.cc:407] Connection header received
0808 10:03:18.084047 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.084047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.084055 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.084125 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.084407 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.084874 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.085545 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.085764 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.085845 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.087953 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.087954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.087954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.087960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.102470 (+ 14510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.125075 (+ 22605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.125077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.125077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.125166 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.125464 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.125464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.125465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.125606 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.125681 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.125914 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.126039 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260808 10:03:18.126148 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.083779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56024)
0808 10:03:18.083833 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.083843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.083878 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.084129 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.084131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.084274 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.084311 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.084313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.084953 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.084954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.085452 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.085454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.085847 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.085848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.085925 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.086326 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.086331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.087860 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.102593 (+ 14733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.102594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.102594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.102856 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.125271 (+ 22415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.125272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.125273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.125378 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.125693 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.125694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.125758 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.125840 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.125992 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:03:18.128429 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:46871
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:45989
--webserver_port=33975
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:03:18.128532 1659 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.162.188:33055: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2b079526c52a49deb541ea4fae2a929f"
member_type: NON_VOTER
last_known_addr {
host: "127.24.162.188"
port: 33055
}
attrs {
promote: true
}
}
dest_uuid: "9c2985209677472790c61be3594f09fe"
cas_config_opid_index: 5
I20260808 10:03:18.128834 1659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } }
I20260808 10:03:18.129267 2466 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2b079526c52a49deb541ea4fae2a929f (127.24.162.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.129690 2666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:18.130448 2466 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.162.188:33055
I20260808 10:03:18.130522 2466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.130580 2466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.130790 2666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:03:18.130810 2606 master.cc:537] Master@127.24.162.188:33055 shutting down...
I20260808 10:03:18.131273 2465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:03:18.132936 1643 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe -> Peer 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055): Couldn't send request to peer 2b079526c52a49deb541ea4fae2a929f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:03:18.134821 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.129366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47068)
0808 10:03:18.130800 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.130812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.130857 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.131491 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.131492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.131494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.131541 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.131543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.132483 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.132484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.133699 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.133703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.134572 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.134573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.134640 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.134661 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.134714 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.134751 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412}
I20260808 10:03:18.135066 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.129456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.24.162.189:45893)
0808 10:03:18.131309 (+ 1853us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.131311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.131319 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:18.131343 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.131343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.131367 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.131431 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.131613 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.132383 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.133802 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.134393 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.134465 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.134537 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.134959 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.135002 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1827}
I20260808 10:03:18.135264 2533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 LEARNER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:03:18.135453 2466 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 10:03:18.135558 2606 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:18.135653 2606 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:18.135722 2606 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: stopping tablet replica
I20260808 10:03:18.136003 2533 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2b079526c52a49deb541ea4fae2a929f (127.24.162.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.137115 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.137197 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: LEARNER
I20260808 10:03:18.137328 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2985209677472790c61be3594f09fe. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: NON_VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: true } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.137451 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: LEARNER
I20260808 10:03:18.137769 2466 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: attempting to promote NON_VOTER 006fee5cbbfa4121a883bf96bcf1f1ff to VOTER
I20260808 10:03:18.137995 2466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } }
I20260808 10:03:18.138586 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.138383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56040)
0808 10:03:18.138514 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.138559 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 10:03:18.138688 2533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 LEARNER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 10:03:18.139083 2465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 10:03:18.139416 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.139288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56056)
0808 10:03:18.139359 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.139385 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:03:18.139488 1643 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe -> Peer 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055): Couldn't send request to peer 2b079526c52a49deb541ea4fae2a929f. Status: Network error: Client connection negotiation failed: client connection to 127.24.162.188:33055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:03:18.139915 2741 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.140089 2533 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.140952 2741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.141049 2741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.141116 2741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.141189 2741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.141418 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.141520 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:03:18.150370 2606 master.cc:552] Master@127.24.162.188:33055 shutdown complete.
I20260808 10:03:18.150427 2606 master_runner.cc:305] Clearing existing system tablet
I20260808 10:03:18.150563 2606 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:18.150614 2606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:18.151392 2606 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:18.151933 2743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.152045 2606 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:18.152091 2606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:18.152114 2606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "2b079526c52a49deb541ea4fae2a929f"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "88363b3b71617cbf2a3aa1f7eb86eee6"
server_key_iv: "076329299ee72cbd4128e217bd1b2a72"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.152840 2606 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:18.154348 2606 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:03:18.154425 2606 log.cc:1196] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 10:03:18.154656 2606 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Deleting consensus metadata
I20260808 10:03:18.154801 2606 master_runner.cc:315] Copying system tablet from 127.24.162.190:45989
I20260808 10:03:18.155550 2606 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.162.190:45989
I20260808 10:03:18.163477 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.155862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:35868)
0808 10:03:18.156009 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.156023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.156091 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.156293 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.156294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.156561 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.156628 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.156631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.157580 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.157582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.158454 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.158457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.158973 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.158974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.159055 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.159787 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.159793 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.160264 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.162235 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.162236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.162236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.162470 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.162716 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.162717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.162717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.162805 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.163120 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.163121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.163180 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.163265 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.163371 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260808 10:03:18.163481 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.156100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.24.162.190:45989)
0808 10:03:18.156139 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.156142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.156151 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:18.156176 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.156176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.156198 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.156258 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.156721 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.157486 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.158568 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.158872 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.158972 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.160395 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.160397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.160397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.160406 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.162138 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.162567 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.162569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.162569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.162635 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.162876 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.162877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.162878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.163044 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.163117 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.163301 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.163394 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:18.164072 1688 tablet_copy_service.cc:140] P 9c2985209677472790c61be3594f09fe: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2b079526c52a49deb541ea4fae2a929f ({username='oryx', principal='oryx/127.24.162.188@KRBTEST.COM'} at 127.0.0.1:35868)
I20260808 10:03:18.164149 1688 tablet_copy_service.cc:161] P 9c2985209677472790c61be3594f09fe: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2b079526c52a49deb541ea4fae2a929f at {username='oryx', principal='oryx/127.24.162.188@KRBTEST.COM'} at 127.0.0.1:35868: session id = 2b079526c52a49deb541ea4fae2a929f-00000000000000000000000000000000
I20260808 10:03:18.164776 1688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:03:18.165306 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:03:18.166719 2606 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: tablet copy: Starting download of 0 data blocks...
I20260808 10:03:18.166857 2606 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: tablet copy: Starting download of 1 WAL segments...
I20260808 10:03:18.167766 2606 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:03:18.168733 1688 tablet_copy_service.cc:342] P 9c2985209677472790c61be3594f09fe: Request end of tablet copy session 2b079526c52a49deb541ea4fae2a929f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.162.188@KRBTEST.COM'} at 127.0.0.1:35868
I20260808 10:03:18.168799 1688 tablet_copy_service.cc:433] P 9c2985209677472790c61be3594f09fe: ending tablet copy session 2b079526c52a49deb541ea4fae2a929f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2b079526c52a49deb541ea4fae2a929f
I20260808 10:03:18.172235 2606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:18.173537 2754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.176069 2757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.176231 2606 server_base.cc:1034] running on GCE node
W20260808 10:03:18.179397 2755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.179612 2606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:18.179949 2606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:18.181084 2606 hybrid_clock.cc:648] HybridClock initialized: now 1786183398181074 us; error 29 us; skew 500 ppm
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:18.182649 2606 init.cc:377] Logged in from keytab as oryx/127.24.162.188@KRBTEST.COM (short username oryx)
I20260808 10:03:18.182953 2606 webserver.cc:460] Webserver started at http://127.24.162.188:45853/ using document root <none> and password file <none>
I20260808 10:03:18.183079 2606 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:18.183116 2606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:18.183884 2606 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:18.184502 2767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.184660 2606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:18.184710 2606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "2b079526c52a49deb541ea4fae2a929f"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "88363b3b71617cbf2a3aa1f7eb86eee6"
server_key_iv: "076329299ee72cbd4128e217bd1b2a72"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.184770 2606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260808 10:03:18.227470 2738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:18.227743 2738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:18.227787 2738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:03:18.228477 2606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260808 10:03:18.229413 2738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:18.229468 2738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:18.229514 2738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:18.230547 2606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:18.231487 2738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:46871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=33975
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:18.231765 2738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:18.232057 2738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:18.234050 2606 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.188:33055
I20260808 10:03:18.234438 2606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
I20260808 10:03:18.235435 2846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.188:33055 every 8 connection(s)
I20260808 10:03:18.235569 2850 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:03:18.236081 2850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Bootstrap starting.
W20260808 10:03:18.238517 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.238607 2843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.238983 2849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.239466 2850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:03:18.239694 2850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: Bootstrap complete.
I20260808 10:03:18.239904 2850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } }
I20260808 10:03:18.239989 2850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b079526c52a49deb541ea4fae2a929f, State: Initialized, Role: LEARNER
I20260808 10:03:18.240041 2850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } }
I20260808 10:03:18.240238 2850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:18.240233 2857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.240471 2857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: LEARNER
I20260808 10:03:18.241364 2738 server_base.cc:1034] running on GCE node
I20260808 10:03:18.241593 2738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:18.241882 2738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:18.243031 2738 hybrid_clock.cc:648] HybridClock initialized: now 1786183398243013 us; error 32 us; skew 500 ppm
I20260808 10:03:18.243291 2877 catalog_manager.cc:1259] Loaded cluster ID: fa7faa2828874bc6a82d97dd571e6ce3
I20260808 10:03:18.243377 2877 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: loading cluster ID for follower catalog manager: success
I20260808 10:03:18.244527 2877 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: acquiring CA information for follower catalog manager: success
I20260808 10:03:18.244814 2877 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:18.245935 2738 init.cc:377] Logged in from keytab as oryx/127.24.162.129@KRBTEST.COM (short username oryx)
I20260808 10:03:18.246322 2738 webserver.cc:460] Webserver started at http://127.24.162.129:33975/ using document root <none> and password file <none>
I20260808 10:03:18.246505 2738 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:18.246574 2738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:18.247979 2738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:18.248607 2884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.248859 2738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:18.248950 2738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "e6745ab226ca4db683d23cb5782e8e21"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "292586404d924e74a892e51efe42287e"
server_key_iv: "fa379e46e0a4a91472e582c4c5e938c6"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.249296 2738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:18.258315 2738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:18.260401 2738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:18.260553 2738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:18.260782 2738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:18.261080 2738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:18.261114 2738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.261153 2738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:18.261179 2738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.267449 2738 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:46871
I20260808 10:03:18.267534 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:46871 every 8 connection(s)
I20260808 10:03:18.267844 2738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:18.275873 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 2738
I20260808 10:03:18.276062 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 1932
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.299405 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.269280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:54149 (local address 127.24.162.189:45893)
0808 10:03:18.269344 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.269346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.269353 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:03:18.270313 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.270314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.270327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.270372 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.271067 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.271561 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.272887 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273101 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273133 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.294718 (+ 21585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.294720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.294721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.294733 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.296839 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.297491 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.297493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.297493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.297559 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298514 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.298516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.298516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.298692 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.298754 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.299210 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.299304 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:03:18.299458 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.268628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59895 (local address 127.24.162.188:33055)
0808 10:03:18.268789 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.268792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.269087 (+ 295us) server_negotiation.cc:407] Connection header received
0808 10:03:18.270336 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.270337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.270351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.270400 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.271093 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.272043 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.272824 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273116 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273181 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.296528 (+ 23347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.296530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.296530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.296540 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.298120 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298573 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.298574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.298575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.298631 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298842 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.298843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.298843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.299006 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.299054 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.299276 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.299361 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260808 10:03:18.300026 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.269077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.24.162.129:54149)
0808 10:03:18.269191 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.269203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.270215 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.270454 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.270458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.270732 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.270963 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.270974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.271669 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.271670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.272773 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.272776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273234 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.273369 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.274020 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.274032 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.294517 (+ 20485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.296967 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.296985 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.296988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.297367 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.298228 (+ 861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.298229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.298417 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.298806 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.298814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.299030 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.299516 (+ 486us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.299762 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260808 10:03:18.300026 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.268633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.24.162.129:59895)
0808 10:03:18.268910 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.268958 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.270206 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.270463 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.270464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.270793 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.270963 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.270969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.272116 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.272117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.272738 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.272740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273205 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.273368 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.274048 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.274052 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.296416 (+ 22364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.298228 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.298229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.298488 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.298706 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.298707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.298765 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.299103 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.299104 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.299143 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.299614 (+ 471us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.299814 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":113,"thread_start_us":204,"threads_started":1}
I20260808 10:03:18.301025 3042 heartbeater.cc:347] Connected to a master server at 127.24.162.188:33055
I20260808 10:03:18.301129 3042 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.301148 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:40741
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:45989
--webserver_port=36669
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:03:18.301174 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.268763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:33121 (local address 127.24.162.190:45989)
0808 10:03:18.268827 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.268830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.269059 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:03:18.270336 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.270337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.270350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.270387 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.271094 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.271845 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.272919 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273221 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273277 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.296317 (+ 23040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.296319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.296319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.296329 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.298638 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.299098 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.299099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.299100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.299158 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.299371 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.299371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.299372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.299515 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.299565 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.300961 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.301053 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 10:03:18.301362 3042 heartbeater.cc:511] Master 127.24.162.188:33055 requested a full tablet report, sending...
I20260808 10:03:18.301504 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.268633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:33121)
0808 10:03:18.268910 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.268958 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.270206 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.270463 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.270464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.270797 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.270964 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.270969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.271920 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.271921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.272840 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.272843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.273279 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.273280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.273371 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.274155 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.274160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.296181 (+ 22021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.298749 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.298751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.298751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.299008 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.299243 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.299244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.299244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.299305 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.300005 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.300006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.300058 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.301268 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.301399 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":9,"thread_start_us":199,"threads_started":1}
I20260808 10:03:18.301971 3041 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:18.302026 3041 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.302094 3041 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:18.302232 3043 heartbeater.cc:347] Connected to a master server at 127.24.162.189:45893
I20260808 10:03:18.302271 3043 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.302322 3043 heartbeater.cc:511] Master 127.24.162.189:45893 requested a full tablet report, sending...
I20260808 10:03:18.302363 1659 ts_manager.cc:194] Re-registered known tserver with Master: e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871)
I20260808 10:03:18.302768 2521 ts_manager.cc:194] Registered new tserver with Master: e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871)
I20260808 10:03:18.302825 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:33121
I20260808 10:03:18.302971 2789 ts_manager.cc:194] Registered new tserver with Master: e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871)
W20260808 10:03:18.384528 3050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:18.384745 3050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:18.384786 3050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:18.386014 3050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:18.386080 3050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:18.386124 3050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:18.387670 3050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:40741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=36669
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:18.387943 3050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:18.388168 3050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:18.393086 3056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.393092 3059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.393159 3050 server_base.cc:1034] running on GCE node
W20260808 10:03:18.393075 3057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.393507 3050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:18.393762 3050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:18.394974 3050 hybrid_clock.cc:648] HybridClock initialized: now 1786183398394971 us; error 39 us; skew 500 ppm
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:18.397507 3050 init.cc:377] Logged in from keytab as oryx/127.24.162.130@KRBTEST.COM (short username oryx)
I20260808 10:03:18.397949 3050 webserver.cc:460] Webserver started at http://127.24.162.130:36669/ using document root <none> and password file <none>
I20260808 10:03:18.398187 3050 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:18.398270 3050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:18.399745 3050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:18.400456 3071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.400622 3050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:18.400743 3050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "2e421faeab2148f4b0cddfd67a561be2"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "c7f40e3f1c52c4917abdf205111f410c"
server_key_iv: "0ca7c57d25bb243398175825b8654b3b"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.401074 3050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:18.414276 3050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:18.416235 3050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:18.416394 3050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:18.416626 3050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:18.416949 3050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:18.417001 3050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.417037 3050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:18.417064 3050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.422901 3050 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40741
I20260808 10:03:18.423013 3224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40741 every 8 connection(s)
I20260808 10:03:18.423350 3050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:18.427081 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 3050
I20260808 10:03:18.427263 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 2134
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
I20260808 10:03:18.436667 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:38693 (local address 127.24.162.189:45893)
0808 10:03:18.424801 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.424804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.424815 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:18.425126 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.425128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.425141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.425210 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.426113 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426867 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.431419 (+ 4552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.431628 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.431663 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.433974 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.433975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.433975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.433982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.435366 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.435764 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.435765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.435765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.435811 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.436080 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.436081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.436081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.436188 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.436225 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.436520 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.436587 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:03:18.437088 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.24.162.130:38693)
0808 10:03:18.424537 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.424547 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.425018 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.425668 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.425668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.425903 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.425998 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.426003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426990 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.426991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.431323 (+ 4332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.431326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.431735 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.431735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.431868 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.432363 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.432372 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.433871 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.435451 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.435457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.435459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.435697 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.435887 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.435887 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.435888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.436018 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.436260 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.436265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.436400 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.436745 (+ 345us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.436900 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":48,"thread_start_us":71,"threads_started":1}
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.445629 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:43597 (local address 127.24.162.190:45989)
0808 10:03:18.424117 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.424119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.424348 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:03:18.425269 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.425270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.425278 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.425320 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.426114 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426889 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.439921 (+ 13032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.440139 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.440228 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.442256 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.442258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.442258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.442266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.444190 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.444662 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.444663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.444663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.444731 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.444961 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.444962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.444963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.445139 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.445213 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.445420 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.445512 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:18.445672 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:43597)
0808 10:03:18.424204 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.424238 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.425013 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.425381 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.425386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.425805 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.425998 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.426000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426993 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.426994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.439813 (+ 12819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.439816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.440231 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.440232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.440303 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.440799 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.440804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.442182 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.444292 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.444293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.444293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.444530 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.444811 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.444812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.444813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.444885 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.445211 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.445212 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.445259 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.445365 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.445506 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":51,"thread_start_us":80,"threads_started":1}
I20260808 10:03:18.446468 3226 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:18.446578 3226 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.446870 3226 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:18.447351 1659 ts_manager.cc:194] Re-registered known tserver with Master: 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741)
I20260808 10:03:18.447835 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:43597
I20260808 10:03:18.448756 3228 heartbeater.cc:347] Connected to a master server at 127.24.162.189:45893
I20260808 10:03:18.448861 3228 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.448932 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:35489 (local address 127.24.162.188:33055)
0808 10:03:18.424842 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.424844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.424854 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:18.425366 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.425367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.425375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.425434 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.426089 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426602 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.438278 (+ 11676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.438523 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.438588 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.445666 (+ 7078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.445668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.445668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.445677 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.447559 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.447956 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.447957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.447957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.448020 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.448254 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.448255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.448256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.448412 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.448485 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.448678 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.448790 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:03:18.448952 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.424571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.24.162.130:35489)
0808 10:03:18.424668 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.424676 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.425024 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.425458 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.425459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.425791 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.425996 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.426000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.426714 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.426715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.438166 (+ 11451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.438169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.438610 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.438611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.438707 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.444101 (+ 5394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.444107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.445556 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.447629 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.447630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.447630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.447861 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.448101 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.448102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.448103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.448181 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.448489 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.448489 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.448528 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.448606 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.448767 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260808 10:03:18.449323 3227 heartbeater.cc:347] Connected to a master server at 127.24.162.188:33055
I20260808 10:03:18.449375 3227 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.449342 3228 heartbeater.cc:511] Master 127.24.162.189:45893 requested a full tablet report, sending...
I20260808 10:03:18.449424 3227 heartbeater.cc:511] Master 127.24.162.188:33055 requested a full tablet report, sending...
I20260808 10:03:18.449823 2521 ts_manager.cc:194] Registered new tserver with Master: 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741)
I20260808 10:03:18.449841 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:44293
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:45989
--webserver_port=42435
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:03:18.450045 2789 ts_manager.cc:194] Registered new tserver with Master: 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741)
W20260808 10:03:18.535715 3238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:18.535986 3238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:18.536026 3238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:18.537433 3238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:18.537499 3238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:18.537539 3238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:18.537762 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.534525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56060)
0808 10:03:18.534623 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.534638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.534726 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.534969 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.534970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.534972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.535028 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.535029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.535939 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.535939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.536715 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.536718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.537542 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.537543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.537607 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.537611 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.537666 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.537704 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:03:18.537788 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.534609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.24.162.188:33055)
0808 10:03:18.534713 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.534716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.534729 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:03:18.534804 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.534805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.534841 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.534943 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.535098 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.535861 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.536814 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.537434 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.537526 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.537603 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.537674 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.537723 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:03:18.538093 2807 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 10:03:18.538291 2857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2985209677472790c61be3594f09fe. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.538408 2857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: LEARNER
I20260808 10:03:18.539312 3238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:44293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=42435
--tserver_master_addrs=127.24.162.190:45989,127.24.162.189:45893,127.24.162.188:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:18.539572 3238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:18.539808 3238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:18.545228 3244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.545243 3248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:18.545249 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:18.545533 3238 server_base.cc:1034] running on GCE node
I20260808 10:03:18.545754 3238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:18.545991 3238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:18.547142 3238 hybrid_clock.cc:648] HybridClock initialized: now 1786183398547132 us; error 31 us; skew 500 ppm
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:18.549713 3238 init.cc:377] Logged in from keytab as oryx/127.24.162.131@KRBTEST.COM (short username oryx)
I20260808 10:03:18.550143 3238 webserver.cc:460] Webserver started at http://127.24.162.131:42435/ using document root <none> and password file <none>
I20260808 10:03:18.550396 3238 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:18.550480 3238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:18.551949 3238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:18.552572 3259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.553678 3238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:18.553767 3238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "a57f521fabc64852a18e8b6b18bda148"
format_stamp: "Formatted at 2026-08-08 10:03:17 on dist-test-slave-nfb5"
server_key: "520d59a95e87971247b45eb85958b56e"
server_key_iv: "02ff2bc711604575fce73d94d4a49aa8"
server_key_version: "encryptionkey@0"
I20260808 10:03:18.554064 3238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:18.569801 3238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:18.571766 3238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:18.571873 3238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:18.572049 3238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:18.572363 3238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:18.572396 3238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.572417 3238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:18.572463 3238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:18.578059 3238 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:44293
I20260808 10:03:18.578125 3419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:44293 every 8 connection(s)
I20260808 10:03:18.578457 3238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:18.585490 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 3238
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.189@KRBTEST.COM
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
Aug 08 10:03:18 dist-test-slave-nfb5 krb5kdc[1597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183398, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.188@KRBTEST.COM
I20260808 10:03:18.590865 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:49201)
0808 10:03:18.579401 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.579446 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.580794 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.580995 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.580999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.581423 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.581758 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.581764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582738 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.582739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.583703 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.583706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.584238 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.584238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.584340 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.584848 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.584853 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.587380 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.589257 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.589265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.589267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.589500 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.589747 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.589748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.589748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.589892 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.590202 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.590209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.590356 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.590454 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.590630 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":55,"thread_start_us":116,"threads_started":1}
I20260808 10:03:18.591341 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:49201 (local address 127.24.162.190:45989)
0808 10:03:18.579307 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.579309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.579533 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:03:18.580878 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.580879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.580892 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.580964 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.581876 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582634 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.583817 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.584107 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.584179 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.587493 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.587494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.587495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.587503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.589160 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.589606 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.589607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.589608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.589675 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.589969 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.589970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.589970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.590102 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.590174 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.591168 (+ 994us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.591249 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:03:18.591677 3421 heartbeater.cc:347] Connected to a master server at 127.24.162.190:45989
I20260808 10:03:18.591763 3421 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.591936 3421 heartbeater.cc:511] Master 127.24.162.190:45989 requested a full tablet report, sending...
I20260808 10:03:18.592459 1659 ts_manager.cc:194] Re-registered known tserver with Master: a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:18.593060 1659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:49201
I20260808 10:03:18.595161 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.24.162.131:34469)
0808 10:03:18.579658 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.579667 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.580624 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.581025 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.581026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.581429 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.581758 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.581764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582738 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.582739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.583711 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.583713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.584131 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.584132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.584205 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.584701 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.584710 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.588785 (+ 4075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.591937 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.591938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.591939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.592257 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.593910 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.593911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.593911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.593974 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.594537 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.594538 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.594585 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.594654 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.594773 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":127,"thread_start_us":66,"threads_started":1}
I20260808 10:03:18.595722 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:34469 (local address 127.24.162.189:45893)
0808 10:03:18.579636 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.579638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.579709 (+ 71us) server_negotiation.cc:407] Connection header received
0808 10:03:18.580762 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.580764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.580778 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.580850 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.581880 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582634 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.583790 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.584054 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.584130 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.589087 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.589089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.589089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.589096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.591855 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.593753 (+ 1898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.593754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.593755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.593823 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.594295 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.594296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.594296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.594439 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.594504 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.595470 (+ 966us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.595594 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:03:18.595834 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.24.162.131:39745)
0808 10:03:18.579621 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.579632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.580622 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.580996 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.581000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.581421 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:18.581758 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.581763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582568 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.582569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.583456 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.583459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.583935 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.583936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.584082 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.585038 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:18.585043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:18.588456 (+ 3413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:18.594277 (+ 5821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.594278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.594278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.594530 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.594836 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:18.594837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:18.594838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:18.594938 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:18.595422 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:18.595423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:18.595478 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.595539 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.595664 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":6,"thread_start_us":115,"threads_started":1}
I20260808 10:03:18.595892 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.579262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:39745 (local address 127.24.162.188:33055)
0808 10:03:18.579310 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.579312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.579693 (+ 381us) server_negotiation.cc:407] Connection header received
0808 10:03:18.580796 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.580798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.580809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.580862 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.581853 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.582470 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.583592 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.583855 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.583924 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.588559 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.588561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.588561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.588570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.594203 (+ 5633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.594634 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.594635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.594636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.594746 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.595024 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.595025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.595025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.595187 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.595655 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.595712 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.595790 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:03:18.596084 3424 heartbeater.cc:347] Connected to a master server at 127.24.162.189:45893
I20260808 10:03:18.596128 3424 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.596204 3424 heartbeater.cc:511] Master 127.24.162.189:45893 requested a full tablet report, sending...
I20260808 10:03:18.596534 2521 ts_manager.cc:194] Registered new tserver with Master: a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:18.596693 3423 heartbeater.cc:347] Connected to a master server at 127.24.162.188:33055
I20260808 10:03:18.596753 3423 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:18.596810 3423 heartbeater.cc:511] Master 127.24.162.188:33055 requested a full tablet report, sending...
I20260808 10:03:18.597118 2789 ts_manager.cc:194] Registered new tserver with Master: a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:18.597707 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.587085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.24.162.189:45893)
0808 10:03:18.588902 (+ 1817us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.588904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.588911 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:03:18.588933 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.588934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.588949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.589012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.589843 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.590557 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.591605 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.591912 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.591981 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.594177 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.594178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.594179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.594187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.596315 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.596744 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.596745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.596745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.596812 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.597056 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.597057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.597057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.597190 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.597295 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.597506 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.597597 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1796,"thread_start_us":61,"threads_started":1}
I20260808 10:03:18.599360 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.587456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.24.162.188:33055)
0808 10:03:18.589084 (+ 1628us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.589086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.589934 (+ 848us) server_negotiation.cc:407] Connection header received
0808 10:03:18.590031 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.590032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.590046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.590092 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.590909 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.591690 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.592784 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.593101 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.593145 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.594626 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.594627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.594627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.594634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.597319 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.598216 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.598217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.598217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.598287 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.598776 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.598777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.598777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.598882 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.599039 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.599166 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.599239 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1607,"thread_start_us":57,"threads_started":1}
I20260808 10:03:18.601946 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.586816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.24.162.190:45989)
0808 10:03:18.586975 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.586978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.586992 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:03:18.587066 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.587067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.587080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.587159 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.588325 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.589160 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.597752 (+ 8592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.598082 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.598163 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.599745 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.599746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.599746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.599754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.600997 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.601347 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.601348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.601348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.601400 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.601521 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.601521 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.601522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.601607 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.601670 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.601736 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.601796 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":108,"threads_started":1}
I20260808 10:03:18.603318 1659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:18.604951 1659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:18.605612 2465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: NON_VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: true } } }
I20260808 10:03:18.605711 2465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.607089 3444 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: attempting to promote NON_VOTER 2b079526c52a49deb541ea4fae2a929f to VOTER
I20260808 10:03:18.607228 3444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:18.607829 2807 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 LEARNER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:03:18.608008 2533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Refusing update from remote peer 9c2985209677472790c61be3594f09fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:03:18.608057 2741 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:03:18.608204 2741 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:03:18.608873 2465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b079526c52a49deb541ea4fae2a929f (127.24.162.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.609004 2533 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b079526c52a49deb541ea4fae2a929f (127.24.162.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.609875 2465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.609910 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.609949 2465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.609977 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:03:18.610028 2465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.610102 2465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.610090 2466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.610189 2466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:18.610064 2807 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b079526c52a49deb541ea4fae2a929f (127.24.162.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.611210 2857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2985209677472790c61be3594f09fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:18.611297 2857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:03:18.615314 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40741 (local address 127.0.0.1:60788)
0808 10:03:18.611869 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.611877 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.611916 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.612223 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.612224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.612226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.612272 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.612273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613427 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.613428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.614143 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.614144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.615062 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.615124 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615128 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.615200 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.615240 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260808 10:03:18.615325 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.24.162.130:40741)
0808 10:03:18.612050 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.612054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.612065 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:18.612104 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.612106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.612122 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.612362 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.612428 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613335 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.614261 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.614991 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615081 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615138 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.615212 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.615247 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
I20260808 10:03:18.615684 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:44293 (local address 127.0.0.1:36566)
0808 10:03:18.611692 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.611702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.611778 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.611966 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.611967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.611971 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.612017 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.612019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613313 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.613314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.614488 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.614490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.615451 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.615521 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615525 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.615557 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.615596 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":418,"thread_start_us":78,"threads_started":1}
I20260808 10:03:18.615792 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46871 (local address 127.0.0.1:47826)
0808 10:03:18.611172 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.611184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.611222 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.612291 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.612292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.612293 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.612341 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.612343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613562 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.613563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.614587 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.614589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.615525 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.615589 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615592 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.615644 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.615686 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:03:18.615800 3460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.24.162.129:46871)
0808 10:03:18.612116 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.612120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.612132 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:18.612172 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.612174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.612190 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.612413 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.612475 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613472 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.614676 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.615429 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615545 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615601 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.615656 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.615703 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":121,"threads_started":1}
I20260808 10:03:18.616051 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.611283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36566 (local address 127.24.162.131:44293)
0808 10:03:18.611472 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.611477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.611774 (+ 297us) server_negotiation.cc:407] Connection header received
0808 10:03:18.611847 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.611849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.611867 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.611954 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.612119 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.613214 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.614588 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.615343 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.615838 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.615908 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.615934 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.615974 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260808 10:03:18.616362 3137 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb0309ca7f74aa1b156b97255812beb (DEFAULT_TABLE table=test-table [id=b3e7a3cfe6cc4d92ac39f7050f671c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:18.616456 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb0309ca7f74aa1b156b97255812beb (DEFAULT_TABLE table=test-table [id=b3e7a3cfe6cc4d92ac39f7050f671c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:18.616559 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb0309ca7f74aa1b156b97255812beb (DEFAULT_TABLE table=test-table [id=b3e7a3cfe6cc4d92ac39f7050f671c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:18.616726 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb0309ca7f74aa1b156b97255812beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:18.616726 3137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb0309ca7f74aa1b156b97255812beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:18.616796 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb0309ca7f74aa1b156b97255812beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:18.619474 3465 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Bootstrap starting.
I20260808 10:03:18.620007 3465 tablet_bootstrap.cc:654] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:18.620267 3465 log.cc:824] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:18.620391 3467 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Bootstrap starting.
I20260808 10:03:18.621042 3467 tablet_bootstrap.cc:654] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:18.621222 3469 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Bootstrap starting.
I20260808 10:03:18.621371 3467 log.cc:824] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:18.621948 3469 tablet_bootstrap.cc:654] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:18.622269 3469 log.cc:824] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:18.622304 3467 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: No bootstrap required, opened a new log
I20260808 10:03:18.622378 3467 ts_tablet_manager.cc:1397] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:18.623065 3465 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: No bootstrap required, opened a new log
I20260808 10:03:18.623140 3465 ts_tablet_manager.cc:1397] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:03:18.623174 3469 tablet_bootstrap.cc:492] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: No bootstrap required, opened a new log
I20260808 10:03:18.623229 3469 ts_tablet_manager.cc:1397] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:18.624234 3467 raft_consensus.cc:348] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.624387 3467 raft_consensus.cc:374] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:18.624432 3467 raft_consensus.cc:729] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e421faeab2148f4b0cddfd67a561be2, State: Initialized, Role: FOLLOWER
I20260808 10:03:18.624552 3467 consensus_queue.cc:260] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.624831 3226 heartbeater.cc:503] Master 127.24.162.190:45989 was elected leader, sending a full tablet report...
I20260808 10:03:18.624838 3467 ts_tablet_manager.cc:1428] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:03:18.625640 3469 raft_consensus.cc:348] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.625825 3469 raft_consensus.cc:374] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:18.625857 3469 raft_consensus.cc:729] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6745ab226ca4db683d23cb5782e8e21, State: Initialized, Role: FOLLOWER
I20260808 10:03:18.625988 3469 consensus_queue.cc:260] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.626021 3465 raft_consensus.cc:348] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.626192 3465 raft_consensus.cc:374] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:18.626245 3469 ts_tablet_manager.cc:1428] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:18.626243 3465 raft_consensus.cc:729] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57f521fabc64852a18e8b6b18bda148, State: Initialized, Role: FOLLOWER
I20260808 10:03:18.626391 3041 heartbeater.cc:503] Master 127.24.162.190:45989 was elected leader, sending a full tablet report...
I20260808 10:03:18.626355 3465 consensus_queue.cc:260] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.626607 3465 ts_tablet_manager.cc:1428] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:18.626713 3421 heartbeater.cc:503] Master 127.24.162.190:45989 was elected leader, sending a full tablet report...
W20260808 10:03:18.674081 3231 tablet.cc:2369] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:18.739343 3475 raft_consensus.cc:482] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:18.739472 3475 raft_consensus.cc:504] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.739799 3475 leader_election.cc:288] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871), a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:18.742866 3460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.740093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:39111 (local address 127.24.162.129:46871)
0808 10:03:18.740140 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.740142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.740150 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:18.740177 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.740177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.740186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.740258 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.740461 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.741131 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.742209 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.742569 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.742646 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.742653 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.742789 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.742813 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:18.742869 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.739986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46871 (local address 127.24.162.130:39111)
0808 10:03:18.740049 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.740060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.740103 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.740276 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.740277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.740278 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.740379 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.740381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.741209 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.741210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.742099 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.742102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.742649 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.742650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.742727 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.742731 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.742776 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.742806 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:03:18.743278 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.739969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:44293 (local address 127.24.162.130:59883)
0808 10:03:18.740025 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:18.740037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:18.740084 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:18.740235 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:18.740236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:18.740238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:18.740344 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.740346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.741066 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.741067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.742100 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:18.742103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.743021 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.743022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:18.743093 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.743096 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:18.743153 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:03:18.743187 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:03:18.743279 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.740055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:59883 (local address 127.24.162.131:44293)
0808 10:03:18.740108 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.740110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.740122 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:18.740149 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.740150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.740165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.740233 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:18.740421 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.740968 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.742188 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.742569 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.742652 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.742660 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.743168 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.743218 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 10:03:18.743305 2977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb0309ca7f74aa1b156b97255812beb" candidate_uuid: "2e421faeab2148f4b0cddfd67a561be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6745ab226ca4db683d23cb5782e8e21" is_pre_election: true
I20260808 10:03:18.743449 2977 raft_consensus.cc:2454] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e421faeab2148f4b0cddfd67a561be2 in term 0.
I20260808 10:03:18.743664 3074 leader_election.cc:302] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e421faeab2148f4b0cddfd67a561be2, e6745ab226ca4db683d23cb5782e8e21; no voters:
I20260808 10:03:18.743649 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb0309ca7f74aa1b156b97255812beb" candidate_uuid: "2e421faeab2148f4b0cddfd67a561be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57f521fabc64852a18e8b6b18bda148" is_pre_election: true
I20260808 10:03:18.743784 3475 raft_consensus.cc:2781] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:18.743803 3359 raft_consensus.cc:2454] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e421faeab2148f4b0cddfd67a561be2 in term 0.
I20260808 10:03:18.743841 3475 raft_consensus.cc:482] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:18.743894 3475 raft_consensus.cc:3037] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:18.744658 3475 raft_consensus.cc:504] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.744819 3475 leader_election.cc:288] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [CANDIDATE]: Term 1 election: Requested vote from peers e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871), a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:18.744974 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb0309ca7f74aa1b156b97255812beb" candidate_uuid: "2e421faeab2148f4b0cddfd67a561be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57f521fabc64852a18e8b6b18bda148"
I20260808 10:03:18.744978 2977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb0309ca7f74aa1b156b97255812beb" candidate_uuid: "2e421faeab2148f4b0cddfd67a561be2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6745ab226ca4db683d23cb5782e8e21"
I20260808 10:03:18.745054 3359 raft_consensus.cc:3037] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:18.745052 2977 raft_consensus.cc:3037] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:18.745791 3359 raft_consensus.cc:2454] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e421faeab2148f4b0cddfd67a561be2 in term 1.
I20260808 10:03:18.745801 2977 raft_consensus.cc:2454] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e421faeab2148f4b0cddfd67a561be2 in term 1.
I20260808 10:03:18.746006 3074 leader_election.cc:302] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e421faeab2148f4b0cddfd67a561be2, e6745ab226ca4db683d23cb5782e8e21; no voters:
I20260808 10:03:18.746114 3475 raft_consensus.cc:2781] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:18.746279 3475 raft_consensus.cc:686] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 1 LEADER]: Becoming Leader. State: Replica: 2e421faeab2148f4b0cddfd67a561be2, State: Running, Role: LEADER
I20260808 10:03:18.746397 3475 consensus_queue.cc:237] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } }
I20260808 10:03:18.747774 1657 catalog_manager.cc:5495] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "2e421faeab2148f4b0cddfd67a561be2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } health_report { overall_health: HEALTHY } } }
I20260808 10:03:18.760273 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.758064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.24.162.130:40741)
0808 10:03:18.758121 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.758123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.758131 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:18.758161 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.758161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.758177 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.758236 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:18.758319 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.758797 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.759566 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.759791 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.759852 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.759972 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:18.760064 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:18.760121 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.760151 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.760168 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:03:18.762041 2977 raft_consensus.cc:1261] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Refusing update from remote peer 2e421faeab2148f4b0cddfd67a561be2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:18.762085 3359 raft_consensus.cc:1261] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Refusing update from remote peer 2e421faeab2148f4b0cddfd67a561be2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:18.762490 3479 consensus_queue.cc:1035] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:18.762616 3475 consensus_queue.cc:1035] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:18.765409 3484 mvcc.cc:205] Tried to move back new op lower bound from 7316207201326981120 to 7316207201265975296. Current Snapshot: MvccSnapshot[applied={T|T < 7316207201326981120}]
I20260808 10:03:18.768345 1659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:18.768424 1659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:18.769913 1659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c2985209677472790c61be3594f09fe: Sending DeleteTablet for 3 replicas of tablet 2bb0309ca7f74aa1b156b97255812beb
I20260808 10:03:18.770275 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb0309ca7f74aa1b156b97255812beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:18 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:47826
I20260808 10:03:18.770514 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb0309ca7f74aa1b156b97255812beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:18 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:36566
I20260808 10:03:18.770591 3137 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb0309ca7f74aa1b156b97255812beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:18 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:60788
I20260808 10:03:18.770836 3497 tablet_replica.cc:330] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: stopping tablet replica
I20260808 10:03:18.770977 3500 tablet_replica.cc:330] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: stopping tablet replica
I20260808 10:03:18.771096 3500 raft_consensus.cc:2229] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:18.771011 3498 tablet_replica.cc:330] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: stopping tablet replica
I20260808 10:03:18.771219 3500 raft_consensus.cc:2258] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:18.771478 3498 raft_consensus.cc:2229] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:18.771137 3497 raft_consensus.cc:2229] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:18.771749 3500 ts_tablet_manager.cc:1905] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:18.771893 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 1620
I20260808 10:03:18.780946 3500 ts_tablet_manager.cc:1918] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:18.781038 3500 log.cc:1196] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/2bb0309ca7f74aa1b156b97255812beb
I20260808 10:03:18.781288 3500 ts_tablet_manager.cc:1939] T 2bb0309ca7f74aa1b156b97255812beb P 2e421faeab2148f4b0cddfd67a561be2: Deleting consensus metadata
I20260808 10:03:18.789264 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.771285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.24.162.188:33055)
0808 10:03:18.771352 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.771355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.781477 (+ 10122us) server_negotiation.cc:407] Connection header received
0808 10:03:18.781908 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.781909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.781925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.781956 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.782623 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.783213 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.784027 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.784232 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.784308 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.785903 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.785905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.785905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.785914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.787600 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.788290 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.788292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.788292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.788353 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.788584 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.788585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.788586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.788708 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.788818 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.788933 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.789021 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 10:03:18.789795 3074 connection.cc:535] server connection from 127.0.0.1:60788 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:03:18.771924 3498 raft_consensus.cc:2258] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:18.771785 3497 raft_consensus.cc:2258] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:18.791611 3498 ts_tablet_manager.cc:1905] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:18.791692 3497 ts_tablet_manager.cc:1905] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:18.793494 3497 ts_tablet_manager.cc:1918] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:18.793582 3497 log.cc:1196] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/2bb0309ca7f74aa1b156b97255812beb
I20260808 10:03:18.793704 3498 ts_tablet_manager.cc:1918] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:18.793830 3498 log.cc:1196] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/2bb0309ca7f74aa1b156b97255812beb
I20260808 10:03:18.793902 3497 ts_tablet_manager.cc:1939] T 2bb0309ca7f74aa1b156b97255812beb P e6745ab226ca4db683d23cb5782e8e21: Deleting consensus metadata
I20260808 10:03:18.794243 3498 ts_tablet_manager.cc:1939] T 2bb0309ca7f74aa1b156b97255812beb P a57f521fabc64852a18e8b6b18bda148: Deleting consensus metadata
W20260808 10:03:18.794283 2885 connection.cc:441] server connection from 127.0.0.1:47826 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47826 (request call id 1) could send its response
W20260808 10:03:18.794705 3264 connection.cc:441] server connection from 127.0.0.1:36566 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36566 (request call id 1) could send its response
I20260808 10:03:18.805642 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.796095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.24.162.189:45893)
0808 10:03:18.796145 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.796147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.796488 (+ 341us) server_negotiation.cc:407] Connection header received
0808 10:03:18.796545 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.796546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.796564 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.796628 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.797673 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.798475 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.800555 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.800898 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.800943 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.802601 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.802603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.802603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.802611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.804446 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.804780 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.804781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.804782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.804839 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.805066 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.805067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.805068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.805216 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.805323 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.805450 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.805542 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:18.806094 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:18.796193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.24.162.188:33055)
0808 10:03:18.796231 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:18.796233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:18.797334 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 10:03:18.797362 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:18.797363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:18.797374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:18.797457 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:18.798636 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.799240 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.801336 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:18.801649 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:18.801732 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:18.803135 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:18.803136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:18.803137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:18.803145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:18.804823 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.805190 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.805191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.805192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.805253 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:18.805492 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:18.805493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:18.805493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:18.805639 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:18.805706 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:18.805923 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:03:18.806004 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:03:19.767125 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:19.766766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:43011)
0808 10:03:19.766955 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:19.767064 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
W20260808 10:03:19.767554 3421 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:45989 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:19.767931 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:19.767664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:38589)
0808 10:03:19.767784 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:19.767872 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260808 10:03:19.768035 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:19.767779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:46045)
0808 10:03:19.767913 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:19.767983 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
W20260808 10:03:19.768231 3226 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:45989 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
W20260808 10:03:19.768251 3041 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:45989 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.270342 3518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2985209677472790c61be3594f09fe)
I20260808 10:03:20.270475 3518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:20.270792 3518 leader_election.cc:288] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055)
I20260808 10:03:20.270876 3520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2985209677472790c61be3594f09fe)
I20260808 10:03:20.271036 3520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:20.271313 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36336)
0808 10:03:20.271165 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.271258 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
W20260808 10:03:20.271463 2505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.271669 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36340)
0808 10:03:20.271513 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.271624 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":40,"threads_started":1}
I20260808 10:03:20.271836 3520 leader_election.cc:288] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189:45893)
W20260808 10:03:20.271905 2770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.271948 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36344)
0808 10:03:20.271894 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.271920 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 10:03:20.272043 2505 leader_election.cc:334] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.272590 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.272369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36350)
0808 10:03:20.272519 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.272558 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":108,"threads_started":1}
W20260808 10:03:20.272742 2770 leader_election.cc:334] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.274436 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.24.162.188:33055)
0808 10:03:20.271333 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:20.271335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:20.271346 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:20.271373 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:20.271374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:20.271387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:20.271459 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:20.271677 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.272473 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.273714 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.274089 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.274175 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:20.274185 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:20.274327 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:03:20.274364 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
I20260808 10:03:20.274432 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:33055 (local address 127.0.0.1:56090)
0808 10:03:20.271211 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.271223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:20.271292 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:20.271521 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:20.271523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:20.271526 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:20.271587 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:20.271590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.272570 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.272572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:20.273591 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:20.273594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.274180 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.274181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:20.274244 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:20.274248 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:20.274319 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:03:20.274363 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260808 10:03:20.274700 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.24.162.189:45893)
0808 10:03:20.272095 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:20.272097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:20.272104 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:03:20.272122 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:20.272123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:20.272131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:20.272311 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:20.272340 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.272983 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.273883 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.274256 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.274334 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:20.274341 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:20.274476 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:03:20.274619 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":85,"threads_started":1}
I20260808 10:03:20.274736 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.271582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45893 (local address 127.0.0.1:47082)
0808 10:03:20.271936 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.271946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:20.271997 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:20.272212 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:20.272212 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:20.272215 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:20.272256 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:20.272257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.273061 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.273062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:20.273784 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:20.273787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:20.274334 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:20.274335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:20.274394 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:20.274397 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:20.274470 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:03:20.274661 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":51,"threads_started":1}
I20260808 10:03:20.274834 2807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b079526c52a49deb541ea4fae2a929f" is_pre_election: true
I20260808 10:03:20.274956 2807 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006fee5cbbfa4121a883bf96bcf1f1ff in term 1.
I20260808 10:03:20.274995 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b079526c52a49deb541ea4fae2a929f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" is_pre_election: true
I20260808 10:03:20.275120 2533 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b079526c52a49deb541ea4fae2a929f in term 1.
I20260808 10:03:20.275146 2507 leader_election.cc:302] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006fee5cbbfa4121a883bf96bcf1f1ff, 2b079526c52a49deb541ea4fae2a929f; no voters: 9c2985209677472790c61be3594f09fe
I20260808 10:03:20.275269 3518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:03:20.275317 3518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2985209677472790c61be3594f09fe)
I20260808 10:03:20.275341 3518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:03:20.275381 2771 leader_election.cc:302] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006fee5cbbfa4121a883bf96bcf1f1ff, 2b079526c52a49deb541ea4fae2a929f; no voters: 9c2985209677472790c61be3594f09fe
I20260808 10:03:20.275481 3520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:03:20.275522 3520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2985209677472790c61be3594f09fe)
I20260808 10:03:20.275543 3520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:03:20.276153 3518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:20.276300 3518 leader_election.cc:288] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 election: Requested vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055)
I20260808 10:03:20.276309 3520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:20.276458 3520 leader_election.cc:288] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 election: Requested vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 006fee5cbbfa4121a883bf96bcf1f1ff (127.24.162.189:45893)
I20260808 10:03:20.276460 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.276375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36362)
0808 10:03:20.276404 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.276433 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 10:03:20.276582 2807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b079526c52a49deb541ea4fae2a929f"
I20260808 10:03:20.276667 2807 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 006fee5cbbfa4121a883bf96bcf1f1ff in current term 2: Already voted for candidate 2b079526c52a49deb541ea4fae2a929f in this term.
I20260808 10:03:20.276634 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b079526c52a49deb541ea4fae2a929f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff"
I20260808 10:03:20.276757 2533 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2b079526c52a49deb541ea4fae2a929f in current term 2: Already voted for candidate 006fee5cbbfa4121a883bf96bcf1f1ff in this term.
I20260808 10:03:20.276926 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.276836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36390)
0808 10:03:20.276869 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.276896 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 10:03:20.276993 2505 leader_election.cc:334] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.277033 2505 leader_election.cc:302] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 006fee5cbbfa4121a883bf96bcf1f1ff; no voters: 2b079526c52a49deb541ea4fae2a929f, 9c2985209677472790c61be3594f09fe
I20260808 10:03:20.277110 3518 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 10:03:20.277225 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.276554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36376)
0808 10:03:20.276972 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.277203 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403}
I20260808 10:03:20.277514 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.277429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36392)
0808 10:03:20.277472 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.277495 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 10:03:20.277590 2770 leader_election.cc:334] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:20.277629 2770 leader_election.cc:302] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b079526c52a49deb541ea4fae2a929f; no voters: 006fee5cbbfa4121a883bf96bcf1f1ff, 9c2985209677472790c61be3594f09fe
I20260808 10:03:20.277748 3520 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 10:03:20.768201 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.767941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:53323)
0808 10:03:20.768114 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.768156 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260808 10:03:20.768765 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.768567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:60153)
0808 10:03:20.768684 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.768714 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":49,"threads_started":1}
I20260808 10:03:20.768812 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:20.768540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:43055)
0808 10:03:20.768663 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:20.768768 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260808 10:03:21.768929 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:21.768666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:44245)
0808 10:03:21.768812 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:21.768856 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 10:03:21.769405 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:21.769170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:36311)
0808 10:03:21.769309 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:21.769344 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260808 10:03:21.769524 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:21.769184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:44357)
0808 10:03:21.769371 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:21.769415 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":50,"threads_started":1}
I20260808 10:03:22.400394 3540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:22.400520 3540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:22.400763 3540 leader_election.cc:288] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055)
I20260808 10:03:22.401019 2807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b079526c52a49deb541ea4fae2a929f" is_pre_election: true
I20260808 10:03:22.401105 2807 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006fee5cbbfa4121a883bf96bcf1f1ff in term 2.
I20260808 10:03:22.401080 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.400896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36700)
0808 10:03:22.401006 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.401041 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":64,"threads_started":1}
I20260808 10:03:22.401305 2507 leader_election.cc:302] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006fee5cbbfa4121a883bf96bcf1f1ff, 2b079526c52a49deb541ea4fae2a929f; no voters:
I20260808 10:03:22.401432 3540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 10:03:22.401484 3540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:22.401531 3540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:03:22.401540 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.401437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36702)
0808 10:03:22.401475 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.401513 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 10:03:22.401623 2505 leader_election.cc:334] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:22.402321 3540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:22.402456 3540 leader_election.cc:288] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 election: Requested vote from peers 9c2985209677472790c61be3594f09fe (127.24.162.190:45989), 2b079526c52a49deb541ea4fae2a929f (127.24.162.188:33055)
I20260808 10:03:22.402633 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.402527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36706)
0808 10:03:22.402579 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.402609 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:03:22.402635 2807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b079526c52a49deb541ea4fae2a929f"
I20260808 10:03:22.402712 2807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:03:22.403050 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.402965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36710)
0808 10:03:22.403000 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.403030 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 10:03:22.403137 2505 leader_election.cc:334] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
I20260808 10:03:22.403560 2807 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 006fee5cbbfa4121a883bf96bcf1f1ff in term 3.
I20260808 10:03:22.403751 2507 leader_election.cc:302] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006fee5cbbfa4121a883bf96bcf1f1ff, 2b079526c52a49deb541ea4fae2a929f; no voters: 9c2985209677472790c61be3594f09fe
I20260808 10:03:22.403849 3540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:03:22.403977 3540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [term 3 LEADER]: Becoming Leader. State: Replica: 006fee5cbbfa4121a883bf96bcf1f1ff, State: Running, Role: LEADER
I20260808 10:03:22.404053 3540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } }
I20260808 10:03:22.404438 3543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006fee5cbbfa4121a883bf96bcf1f1ff. Latest consensus state: current_term: 3 leader_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:22.404541 3543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:22.404727 3545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:22.405294 3545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:22.405393 3545 catalog_manager.cc:1259] Loaded cluster ID: fa7faa2828874bc6a82d97dd571e6ce3
I20260808 10:03:22.405428 3545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:22.405613 3545 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:22.405699 3545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Loaded TSK: 0
I20260808 10:03:22.405843 3545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:22.453697 2520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:22.455466 2520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:22.456555 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.456425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36732)
0808 10:03:22.456483 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.456522 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:03:22.456544 2807 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [term 3 FOLLOWER]: Refusing update from remote peer 006fee5cbbfa4121a883bf96bcf1f1ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 10:03:22.456763 3543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 10:03:22.456921 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.456815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.0.0.1:36744)
0808 10:03:22.456864 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.456896 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:03:22.456993 2505 consensus_peers.cc:487] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff -> Peer 9c2985209677472790c61be3594f09fe (127.24.162.190:45989): Couldn't send request to peer 9c2985209677472790c61be3594f09fe. Status: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:03:22.457479 3549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006fee5cbbfa4121a883bf96bcf1f1ff. Latest consensus state: current_term: 3 leader_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:22.457597 3549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:03:22.458108 3540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:22.458204 3540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:22.458155 3549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9c2985209677472790c61be3594f09fe" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45989 } } peers { permanent_uuid: "006fee5cbbfa4121a883bf96bcf1f1ff" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45893 } attrs { promote: false } } peers { permanent_uuid: "2b079526c52a49deb541ea4fae2a929f" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 33055 } attrs { promote: false } } }
I20260808 10:03:22.458240 3549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b079526c52a49deb541ea4fae2a929f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:03:22.463301 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:44293 (local address 127.0.0.1:36574)
0808 10:03:22.460400 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.460410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:22.460454 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:22.460615 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:22.460616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:22.460618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:22.460659 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.460660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461538 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.461539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.462304 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.462306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.463055 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.463056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.463115 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.463119 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:22.463175 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:03:22.463214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":33,"thread_start_us":78,"threads_started":1}
I20260808 10:03:22.463303 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.24.162.131:44293)
0808 10:03:22.460489 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.460491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.460499 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:22.460525 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.460526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.460537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.460608 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:22.460746 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461469 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.462397 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.462977 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.463051 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.463059 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.463189 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.463227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260808 10:03:22.463687 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40741 (local address 127.0.0.1:60810)
0808 10:03:22.460073 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.460087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:22.460144 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:22.460407 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:22.460408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:22.460411 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:22.460474 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.460476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461482 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.461482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.462633 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.462636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.463479 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.463480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.463543 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.463548 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:03:22.463576 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 10:03:22.463627 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:03:22.463872 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46871 (local address 127.0.0.1:47834)
0808 10:03:22.460701 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.460711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:22.460762 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:22.460897 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:22.460897 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:22.460899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:22.460946 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.460948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461616 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.461617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.462797 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.462799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.463594 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.463595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.463657 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.463661 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:22.463743 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:03:22.463798 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":195,"thread_start_us":65,"threads_started":1}
I20260808 10:03:22.463892 3328 tablet_service.cc:1467] Processing CreateTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 (DEFAULT_TABLE table=test-table [id=5176e9af917b4ecc9be0146a5a414807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:22.463961 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.24.162.129:46871)
0808 10:03:22.460340 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.460342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.460754 (+ 412us) server_negotiation.cc:407] Connection header received
0808 10:03:22.460820 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.460822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.460833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.460896 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:22.460997 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461529 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.462883 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.463489 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.463582 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.463592 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.463839 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.463885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":77,"threads_started":1}
I20260808 10:03:22.464051 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f97f4c2f7b96431590c4e07d39ad9ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:22.464145 2948 tablet_service.cc:1467] Processing CreateTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 (DEFAULT_TABLE table=test-table [id=5176e9af917b4ecc9be0146a5a414807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:22.464282 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f97f4c2f7b96431590c4e07d39ad9ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:22.465463 3561 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Bootstrap starting.
I20260808 10:03:22.465641 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.460106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.24.162.130:40741)
0808 10:03:22.460226 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.460228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.460239 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:22.460271 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.460272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.460287 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.460539 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:22.460601 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.461395 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.462705 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.463303 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.465489 (+ 2186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.465501 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.465521 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.465565 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260808 10:03:22.465665 3562 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: Bootstrap starting.
I20260808 10:03:22.465857 3137 tablet_service.cc:1467] Processing CreateTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 (DEFAULT_TABLE table=test-table [id=5176e9af917b4ecc9be0146a5a414807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:22.465965 3561 tablet_bootstrap.cc:654] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:22.465982 3137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f97f4c2f7b96431590c4e07d39ad9ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:22.466112 3562 tablet_bootstrap.cc:654] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:22.466701 3562 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: No bootstrap required, opened a new log
I20260808 10:03:22.466696 3561 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: No bootstrap required, opened a new log
I20260808 10:03:22.466749 3561 ts_tablet_manager.cc:1397] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:22.466750 3562 ts_tablet_manager.cc:1397] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:22.466863 3562 raft_consensus.cc:348] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.466877 3561 raft_consensus.cc:348] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.466956 3562 raft_consensus.cc:374] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:22.466979 3562 raft_consensus.cc:729] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6745ab226ca4db683d23cb5782e8e21, State: Initialized, Role: FOLLOWER
I20260808 10:03:22.466977 3561 raft_consensus.cc:374] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:22.467000 3561 raft_consensus.cc:729] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57f521fabc64852a18e8b6b18bda148, State: Initialized, Role: FOLLOWER
I20260808 10:03:22.467033 3562 consensus_queue.cc:260] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.467134 3561 consensus_queue.cc:260] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.467211 3562 ts_tablet_manager.cc:1428] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:22.467347 3561 ts_tablet_manager.cc:1428] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:22.467793 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.467554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:42743)
0808 10:03:22.467695 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.467747 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260808 10:03:22.468043 3566 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Bootstrap starting.
I20260808 10:03:22.468441 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.467573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:60431)
0808 10:03:22.468352 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.468389 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":69,"threads_started":1}
I20260808 10:03:22.468443 3566 tablet_bootstrap.cc:654] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:22.468726 3424 heartbeater.cc:503] Master 127.24.162.189:45893 was elected leader, sending a full tablet report...
I20260808 10:03:22.468859 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.468729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:48383)
0808 10:03:22.468764 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.468795 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 10:03:22.468961 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.468870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:42241)
0808 10:03:22.468910 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.468933 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:03:22.469106 3043 heartbeater.cc:503] Master 127.24.162.189:45893 was elected leader, sending a full tablet report...
I20260808 10:03:22.469379 3566 tablet_bootstrap.cc:492] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: No bootstrap required, opened a new log
I20260808 10:03:22.469447 3566 ts_tablet_manager.cc:1397] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:22.469559 3566 raft_consensus.cc:348] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.469648 3566 raft_consensus.cc:374] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:22.469682 3566 raft_consensus.cc:729] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e421faeab2148f4b0cddfd67a561be2, State: Initialized, Role: FOLLOWER
I20260808 10:03:22.469734 3566 consensus_queue.cc:260] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.469894 3566 ts_tablet_manager.cc:1428] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:22.470249 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.470032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:37785)
0808 10:03:22.470172 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.470212 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":96,"threads_started":1}
I20260808 10:03:22.470290 3228 heartbeater.cc:503] Master 127.24.162.189:45893 was elected leader, sending a full tablet report...
I20260808 10:03:22.470688 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.470530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:43643)
0808 10:03:22.470636 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.470662 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 10:03:22.521253 3044 tablet.cc:2369] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:03:22.581877 3425 tablet.cc:2369] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:22.638584 3565 raft_consensus.cc:482] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:22.638762 3565 raft_consensus.cc:504] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.639190 3565 leader_election.cc:288] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741), a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:22.642813 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.639510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:41203 (local address 127.24.162.130:40741)
0808 10:03:22.639566 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.639568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.639582 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:03:22.639622 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.639623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.639633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.639713 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:22.639922 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.640552 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642054 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.642438 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642496 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.642504 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.642689 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.642733 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:03:22.642930 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.639600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:44335 (local address 127.24.162.131:44293)
0808 10:03:22.639669 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.639672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.639799 (+ 127us) server_negotiation.cc:407] Connection header received
0808 10:03:22.639833 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.639835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.639846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.639908 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:22.640058 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.640729 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642055 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.642523 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642602 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.642613 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.642794 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.642834 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:03:22.642982 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.639520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:44293 (local address 127.24.162.129:44335)
0808 10:03:22.639722 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.639733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:22.639794 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:22.639921 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:22.639922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:22.639924 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:22.639991 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.639992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.640808 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.640809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.641963 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.641966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.642627 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.642708 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.642712 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:22.642770 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:03:22.642809 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":19,"thread_start_us":130,"threads_started":1}
I20260808 10:03:22.643236 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f97f4c2f7b96431590c4e07d39ad9ec4" candidate_uuid: "e6745ab226ca4db683d23cb5782e8e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57f521fabc64852a18e8b6b18bda148" is_pre_election: true
I20260808 10:03:22.642822 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.639399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40741 (local address 127.24.162.129:41203)
0808 10:03:22.639469 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.639483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:22.639564 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:22.639719 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:22.639725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:22.639727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:22.639836 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.639837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.640636 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.640647 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.641964 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:22.641967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.642517 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.642519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:22.642606 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.642610 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:22.642671 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:03:22.642716 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":21}
I20260808 10:03:22.643333 3359 raft_consensus.cc:2454] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6745ab226ca4db683d23cb5782e8e21 in term 0.
I20260808 10:03:22.643570 3160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f97f4c2f7b96431590c4e07d39ad9ec4" candidate_uuid: "e6745ab226ca4db683d23cb5782e8e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e421faeab2148f4b0cddfd67a561be2" is_pre_election: true
I20260808 10:03:22.643651 2887 leader_election.cc:302] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57f521fabc64852a18e8b6b18bda148, e6745ab226ca4db683d23cb5782e8e21; no voters:
I20260808 10:03:22.643733 3160 raft_consensus.cc:2454] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6745ab226ca4db683d23cb5782e8e21 in term 0.
I20260808 10:03:22.643827 3565 raft_consensus.cc:2781] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:22.643882 3565 raft_consensus.cc:482] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:22.643908 3565 raft_consensus.cc:3037] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:22.644701 3565 raft_consensus.cc:504] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.644850 3565 leader_election.cc:288] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [CANDIDATE]: Term 1 election: Requested vote from peers 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741), a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293)
I20260808 10:03:22.645004 3160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f97f4c2f7b96431590c4e07d39ad9ec4" candidate_uuid: "e6745ab226ca4db683d23cb5782e8e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e421faeab2148f4b0cddfd67a561be2"
I20260808 10:03:22.645052 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f97f4c2f7b96431590c4e07d39ad9ec4" candidate_uuid: "e6745ab226ca4db683d23cb5782e8e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57f521fabc64852a18e8b6b18bda148"
I20260808 10:03:22.645092 3160 raft_consensus.cc:3037] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:22.645145 3359 raft_consensus.cc:3037] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:22.645826 3160 raft_consensus.cc:2454] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6745ab226ca4db683d23cb5782e8e21 in term 1.
I20260808 10:03:22.645849 3359 raft_consensus.cc:2454] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6745ab226ca4db683d23cb5782e8e21 in term 1.
I20260808 10:03:22.646015 2887 leader_election.cc:302] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57f521fabc64852a18e8b6b18bda148, e6745ab226ca4db683d23cb5782e8e21; no voters:
I20260808 10:03:22.646170 3565 raft_consensus.cc:2781] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:22.646328 3565 raft_consensus.cc:686] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [term 1 LEADER]: Becoming Leader. State: Replica: e6745ab226ca4db683d23cb5782e8e21, State: Running, Role: LEADER
I20260808 10:03:22.646409 3565 consensus_queue.cc:237] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } }
I20260808 10:03:22.646737 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.646573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:43729)
0808 10:03:22.646662 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.646702 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 10:03:22.647310 2521 catalog_manager.cc:5495] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "e6745ab226ca4db683d23cb5782e8e21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6745ab226ca4db683d23cb5782e8e21" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:22.691223 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.688382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.24.162.129:46871)
0808 10:03:22.688459 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:22.688462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:22.688472 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:22.688522 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:22.688523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:22.688543 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:22.688628 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:22.688776 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.689353 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.690171 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:22.690416 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:22.690510 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:22.690710 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:22.690818 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:22.690909 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:22.691125 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 10:03:22.691157 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:03:22.692502 3359 raft_consensus.cc:1261] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Refusing update from remote peer e6745ab226ca4db683d23cb5782e8e21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:22.692560 3160 raft_consensus.cc:1261] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Refusing update from remote peer e6745ab226ca4db683d23cb5782e8e21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:22.692871 3565 consensus_queue.cc:1035] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57f521fabc64852a18e8b6b18bda148" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:22.693054 3582 consensus_queue.cc:1035] T f97f4c2f7b96431590c4e07d39ad9ec4 P e6745ab226ca4db683d23cb5782e8e21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e421faeab2148f4b0cddfd67a561be2" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:22.693868 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.693719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:40829)
0808 10:03:22.693771 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.693805 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:03:22.694653 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.694492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:37669)
0808 10:03:22.694581 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.694615 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 10:03:22.695104 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.694944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:58159)
0808 10:03:22.695042 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.695076 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 10:03:22.695637 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.695506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.129:51773)
0808 10:03:22.695577 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.695611 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:03:22.696154 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.696001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.131:37405)
0808 10:03:22.696090 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.696121 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 10:03:22.696746 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:22.696654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45989 (local address 127.24.162.130:48669)
0808 10:03:22.696695 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.696720 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:45989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 10:03:22.697391 2520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:22.697496 2520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:22.698813 2520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006fee5cbbfa4121a883bf96bcf1f1ff: Sending DeleteTablet for 3 replicas of tablet f97f4c2f7b96431590c4e07d39ad9ec4
I20260808 10:03:22.699165 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:22 UTC) from {username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:47834
I20260808 10:03:22.699230 3137 tablet_service.cc:1514] Processing DeleteTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:22 UTC) from {username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:60810
I20260808 10:03:22.699477 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet f97f4c2f7b96431590c4e07d39ad9ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:22 UTC) from {username='oryx', principal='oryx/127.24.162.189@KRBTEST.COM'} at 127.0.0.1:36574
I20260808 10:03:22.699520 3602 tablet_replica.cc:330] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: stopping tablet replica
I20260808 10:03:22.699529 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 2738
I20260808 10:03:22.699582 3602 raft_consensus.cc:2229] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:22.705358 3602 raft_consensus.cc:2258] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:22.711248 3603 tablet_replica.cc:330] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: stopping tablet replica
I20260808 10:03:22.711354 3603 raft_consensus.cc:2229] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:22.711416 3603 raft_consensus.cc:2258] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:22.711731 3603 ts_tablet_manager.cc:1905] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:22.712735 3602 ts_tablet_manager.cc:1905] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:22.713281 3603 ts_tablet_manager.cc:1918] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:22.713356 3603 log.cc:1196] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/f97f4c2f7b96431590c4e07d39ad9ec4
I20260808 10:03:22.713608 3603 ts_tablet_manager.cc:1939] T f97f4c2f7b96431590c4e07d39ad9ec4 P a57f521fabc64852a18e8b6b18bda148: Deleting consensus metadata
I20260808 10:03:22.714021 2505 catalog_manager.cc:4841] TS a57f521fabc64852a18e8b6b18bda148 (127.24.162.131:44293): tablet f97f4c2f7b96431590c4e07d39ad9ec4 (table test-table [id=5176e9af917b4ecc9be0146a5a414807]) successfully deleted
I20260808 10:03:22.714433 3602 ts_tablet_manager.cc:1918] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:22.714497 3602 log.cc:1196] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/f97f4c2f7b96431590c4e07d39ad9ec4
I20260808 10:03:22.714761 3602 ts_tablet_manager.cc:1939] T f97f4c2f7b96431590c4e07d39ad9ec4 P 2e421faeab2148f4b0cddfd67a561be2: Deleting consensus metadata
W20260808 10:03:22.714807 3557 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:22.714648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46871 (local address 127.0.0.1:47840)
0808 10:03:22.714697 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:22.714734 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:46871: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 10:03:22.714982 2502 catalog_manager.cc:4584] TS e6745ab226ca4db683d23cb5782e8e21 (127.24.162.129:46871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f97f4c2f7b96431590c4e07d39ad9ec4: Network error: Client connection negotiation failed: client connection to 127.24.162.129:46871: connect: Connection reset by peer (error 104)
I20260808 10:03:22.715116 2505 catalog_manager.cc:4841] TS 2e421faeab2148f4b0cddfd67a561be2 (127.24.162.130:40741): tablet f97f4c2f7b96431590c4e07d39ad9ec4 (table test-table [id=5176e9af917b4ecc9be0146a5a414807]) successfully deleted
I20260808 10:03:22.715179 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 3050
I20260808 10:03:22.738790 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 3238
I20260808 10:03:22.752658 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 2337
I20260808 10:03:22.761554 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 2606
2026-08-08T10:03:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8584 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:22 dist-test-slave-nfb5 krb5kdc[3608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:22 dist-test-slave-nfb5 krb5kdc[3608](info): set up 2 sockets
Aug 08 10:03:22 dist-test-slave-nfb5 krb5kdc[3608](info): commencing operation
krb5kdc: starting...
W20260808 10:03:23.991262 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab.
Entry for principal oryx/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab.
I20260808 10:03:24.128059 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37897
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab
--principal=oryx/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:24.212916 3626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:24.213116 3626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:24.213138 3626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:24.214306 3626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:24.214351 3626 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:03:24.214365 3626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:24.214378 3626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:24.214433 3626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:24.214450 3626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:24.215988 3626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:37897
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.190.keytab
--principal=oryx/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:24.216259 3626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:24.216495 3626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:24.221721 3626 server_base.cc:1034] running on GCE node
W20260808 10:03:24.221743 3634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.221740 3632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.221848 3636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:24.222162 3626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:24.222455 3626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:24.223646 3626 hybrid_clock.cc:648] HybridClock initialized: now 1786183404223621 us; error 39 us; skew 500 ppm
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:24.226347 3626 init.cc:377] Logged in from keytab as oryx/127.24.162.190@KRBTEST.COM (short username oryx)
I20260808 10:03:24.226902 3626 webserver.cc:460] Webserver started at http://127.24.162.190:35137/ using document root <none> and password file <none>
I20260808 10:03:24.227231 3626 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:24.227305 3626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:24.227473 3626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:24.228626 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "3f16fddc91594e47a4d56f703dd6654f"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "245927e3d73a37129a1a8ceebf67d869"
server_key_iv: "4d9968fad42fc0c550bc13aaa730abe0"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.229033 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "3f16fddc91594e47a4d56f703dd6654f"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "245927e3d73a37129a1a8ceebf67d869"
server_key_iv: "4d9968fad42fc0c550bc13aaa730abe0"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.230476 3626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:24.231367 3646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.231547 3626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.231698 3626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "3f16fddc91594e47a4d56f703dd6654f"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "245927e3d73a37129a1a8ceebf67d869"
server_key_iv: "4d9968fad42fc0c550bc13aaa730abe0"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.231787 3626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:24.240454 3626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:24.242403 3626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:24.242584 3626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:24.247824 3626 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:37897
I20260808 10:03:24.247876 3735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:37897 every 8 connection(s)
I20260808 10:03:24.248224 3626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:24.248883 3736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:24.251147 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Bootstrap starting.
I20260808 10:03:24.251732 3736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:24.251986 3736 log.cc:824] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:24.252642 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: No bootstrap required, opened a new log
I20260808 10:03:24.253028 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 3626
I20260808 10:03:24.253134 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:24.253333 25226 external_mini_cluster.cc:1428] Setting key 0e730dc9fd101d38b030a6c4954df243
I20260808 10:03:24.255373 3736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } }
I20260808 10:03:24.255505 3736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:24.255527 3736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f16fddc91594e47a4d56f703dd6654f, State: Initialized, Role: FOLLOWER
I20260808 10:03:24.255702 3736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } }
I20260808 10:03:24.255795 3736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:24.255831 3736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:24.255882 3736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:24.256590 3736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } }
I20260808 10:03:24.256685 3736 leader_election.cc:302] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f16fddc91594e47a4d56f703dd6654f; no voters:
I20260808 10:03:24.256867 3736 leader_election.cc:288] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:24.256942 3741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:24.257084 3741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f16fddc91594e47a4d56f703dd6654f, State: Running, Role: LEADER
I20260808 10:03:24.257144 3736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:24.257174 3741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } }
I20260808 10:03:24.257615 3742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f16fddc91594e47a4d56f703dd6654f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } } }
I20260808 10:03:24.257728 3742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:24.257968 3752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:24.257989 3743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f16fddc91594e47a4d56f703dd6654f. Latest consensus state: current_term: 1 leader_uuid: "3f16fddc91594e47a4d56f703dd6654f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f16fddc91594e47a4d56f703dd6654f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37897 } } }
I20260808 10:03:24.258214 3743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:24.258550 3752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:24.260285 3752 catalog_manager.cc:1347] Generated new cluster ID: ba6534dba6cd47e7825b8aa3825902d7
I20260808 10:03:24.260376 3752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:24.266242 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.253888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.24.162.190:37897)
0808 10:03:24.254078 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.254086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.254112 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:03:24.255368 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.255376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.255431 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.255603 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.256432 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.257866 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.258989 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.259755 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.259860 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.262333 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.262337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.262337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.262350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.264610 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.265027 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.265030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.265031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.265141 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.265368 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.265369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.265369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.265614 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.265815 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.265885 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.266030 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260808 10:03:24.286365 3752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:24.287194 3752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:24.291467 3752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Generated new TSK 0
I20260808 10:03:24.291663 3752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
I20260808 10:03:24.320189 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37897
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:24.402078 3772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:24.402304 3772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:24.402345 3772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:24.403541 3772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:24.403606 3772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:24.403642 3772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:03:24.403680 3772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:24.405172 3772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:24.405455 3772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:24.405683 3772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:24.410669 3777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.410732 3779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.410683 3784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:24.410727 3772 server_base.cc:1034] running on GCE node
I20260808 10:03:24.411078 3772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:24.411326 3772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:24.412496 3772 hybrid_clock.cc:648] HybridClock initialized: now 1786183404412479 us; error 52 us; skew 500 ppm
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:24.414989 3772 init.cc:377] Logged in from keytab as oryx/127.24.162.129@KRBTEST.COM (short username oryx)
I20260808 10:03:24.415452 3772 webserver.cc:460] Webserver started at http://127.24.162.129:42177/ using document root <none> and password file <none>
I20260808 10:03:24.415714 3772 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:24.415782 3772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:24.415940 3772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:24.417032 3772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "33fec224d4174ea4a82092f759fd8a59"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "440192e4aae46a4a429a6d198db7a59d"
server_key_iv: "996bcffb357c2e83add3d9aa15d1ef6e"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.417407 3772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "33fec224d4174ea4a82092f759fd8a59"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "440192e4aae46a4a429a6d198db7a59d"
server_key_iv: "996bcffb357c2e83add3d9aa15d1ef6e"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.418738 3772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:24.419538 3792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.419718 3772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:24.419816 3772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "33fec224d4174ea4a82092f759fd8a59"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "440192e4aae46a4a429a6d198db7a59d"
server_key_iv: "996bcffb357c2e83add3d9aa15d1ef6e"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.419894 3772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:24.439294 3772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:24.441084 3772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:24.441246 3772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:24.441823 3772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:24.441874 3772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.441908 3772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:24.441928 3772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:24.449635 3772 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:44559
I20260808 10:03:24.450075 3772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:24.450069 3960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:44559 every 8 connection(s)
I20260808 10:03:24.452148 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.442197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:52033 (local address 127.24.162.190:37897)
0808 10:03:24.442264 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.442267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.442455 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:03:24.443019 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.443020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.443034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.443093 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.443694 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.444268 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.445416 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.445745 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.445828 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.449028 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.449029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.449030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.449040 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.450868 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.451323 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.451324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.451324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.451379 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.451594 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.451594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.451594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.451703 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.451763 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.452014 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.452083 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:03:24.452310 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.442179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.129:52033)
0808 10:03:24.442333 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.442369 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.442870 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.443168 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.443172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.443410 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.443589 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.443593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.444410 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.444412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.445300 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.445304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.445841 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.445842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.446007 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.446535 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.446545 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.448940 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.451006 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.451011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.451012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.451253 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.451439 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.451439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.451440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.451540 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.451763 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.451768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.451899 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.451982 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.452099 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260808 10:03:24.452827 3961 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37897
I20260808 10:03:24.452912 3961 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:24.453101 3961 heartbeater.cc:511] Master 127.24.162.190:37897 requested a full tablet report, sending...
I20260808 10:03:24.453673 3669 ts_manager.cc:194] Registered new tserver with Master: 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559)
I20260808 10:03:24.454517 3669 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:52033
I20260808 10:03:24.455886 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 3772
I20260808 10:03:24.455978 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:24.456192 25226 external_mini_cluster.cc:1428] Setting key 6e2bb8ce80ce406068b04733a79d8fb7
I20260808 10:03:24.460655 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.453899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.129:57191)
0808 10:03:24.454070 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.454079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.454128 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.454251 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.454251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.454401 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.454455 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.454457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.455313 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.455314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.455894 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.455895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.456429 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.456430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.456521 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.456986 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.456991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.457426 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.459682 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.459683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.459684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.459851 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.460074 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.460075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.460075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.460125 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.460415 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.460416 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.460453 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.460506 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.460579 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260808 10:03:24.460808 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.453968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:57191 (local address 127.24.162.190:37897)
0808 10:03:24.454007 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.454009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.454135 (+ 126us) server_negotiation.cc:407] Connection header received
0808 10:03:24.454176 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.454177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.454190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.454248 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.454518 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.455176 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.456059 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.456339 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.456433 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.457560 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.457561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.457561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.457568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.459566 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.459930 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.459931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.459931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.460000 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.460188 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.460189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.460189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.460345 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.460423 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.460536 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.460628 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
WARNING: no policy specified for oryx/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
I20260808 10:03:24.484354 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37897
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:24.567708 3971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:24.567909 3971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:24.567934 3971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:24.569219 3971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:24.569274 3971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:24.569296 3971 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:03:24.569314 3971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:24.571017 3971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:24.571213 3971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:24.571396 3971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:24.576279 3979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.576288 3981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:24.576443 3971 server_base.cc:1034] running on GCE node
W20260808 10:03:24.576288 3985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:24.576761 3971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:24.577018 3971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:24.578176 3971 hybrid_clock.cc:648] HybridClock initialized: now 1786183404578172 us; error 38 us; skew 500 ppm
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:24.580721 3971 init.cc:377] Logged in from keytab as oryx/127.24.162.130@KRBTEST.COM (short username oryx)
I20260808 10:03:24.581223 3971 webserver.cc:460] Webserver started at http://127.24.162.130:39391/ using document root <none> and password file <none>
I20260808 10:03:24.581475 3971 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:24.581543 3971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:24.581684 3971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:24.582713 3971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "e2e6abf8c4b0472abfbbe86128b5c540"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "ae7b1e3c450da1c190e67cdfe0bd1b8f"
server_key_iv: "ac1701e3de4892dd215b00753f8c8ebb"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.583091 3971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "e2e6abf8c4b0472abfbbe86128b5c540"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "ae7b1e3c450da1c190e67cdfe0bd1b8f"
server_key_iv: "ac1701e3de4892dd215b00753f8c8ebb"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.584422 3971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:24.585167 3995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.585353 3971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:24.585451 3971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "e2e6abf8c4b0472abfbbe86128b5c540"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "ae7b1e3c450da1c190e67cdfe0bd1b8f"
server_key_iv: "ac1701e3de4892dd215b00753f8c8ebb"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.585528 3971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:24.607147 3971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:24.609305 3971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:24.609469 3971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:24.610059 3971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:24.610116 3971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.610162 3971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:24.610206 3971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:24.617046 3971 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:36481
I20260808 10:03:24.617129 4151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:36481 every 8 connection(s)
I20260808 10:03:24.617421 3971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:24.620193 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 3971
I20260808 10:03:24.620301 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:24.620503 25226 external_mini_cluster.cc:1428] Setting key 845134166f278bebbacc56f5ca9731a5
I20260808 10:03:24.620735 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.610526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.130:48537)
0808 10:03:24.610691 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.610727 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.611319 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.611587 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.611590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.611892 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.612080 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.612085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.613134 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.613136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.614588 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.614592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.615141 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.615142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.615299 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.615792 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.615802 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.617400 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.619223 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.619230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.619233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.619454 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.619670 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.619671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.619672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.619790 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.620065 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.620072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.620206 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.620322 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.620472 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260808 10:03:24.621059 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.610546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:48537 (local address 127.24.162.190:37897)
0808 10:03:24.610607 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.610609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.610810 (+ 201us) server_negotiation.cc:407] Connection header received
0808 10:03:24.611431 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.611433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.611446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.611496 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.612175 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.613024 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.614698 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.615051 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.615132 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.617497 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.617498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.617499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.617506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.619119 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.619531 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.619533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.619533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.619601 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.619868 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.619869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.619870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.619989 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.620061 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.620894 (+ 833us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.620981 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:03:24.623307 4152 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37897
I20260808 10:03:24.623415 4152 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:24.623648 4152 heartbeater.cc:511] Master 127.24.162.190:37897 requested a full tablet report, sending...
I20260808 10:03:24.623988 3669 ts_manager.cc:194] Registered new tserver with Master: e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481)
I20260808 10:03:24.624512 3669 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:48537
I20260808 10:03:24.628723 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.623024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:44513 (local address 127.24.162.190:37897)
0808 10:03:24.623084 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.623086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.623096 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:24.623117 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.623118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.623127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.623199 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.623561 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.624157 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.624947 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.625122 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.625214 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.626056 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.626058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.626058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.626065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.627583 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.627917 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.627918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.627918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.627973 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.628170 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.628170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.628171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.628324 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.628399 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.628565 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.628636 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":36}
I20260808 10:03:24.628732 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.622386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.130:44513)
0808 10:03:24.622539 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.622552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.622604 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.623215 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.623217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.623423 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.623490 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.623492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.624249 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.624250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.624865 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.624868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.625218 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.625220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.625290 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.625691 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.625695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.625968 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.627661 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.627662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.627663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.627844 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.628041 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.628042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.628043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.628101 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.628402 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.628402 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.628448 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.628523 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.628627 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
I20260808 10:03:24.649530 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37897
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:24.732760 4162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:24.732963 4162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:24.732986 4162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:24.734145 4162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:24.734185 4162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:24.734205 4162 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:03:24.734223 4162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:24.735853 4162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.162.190:37897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:24.736045 4162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:24.736235 4162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:24.741333 4172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.741307 4171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:24.741307 4176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:24.741344 4162 server_base.cc:1034] running on GCE node
I20260808 10:03:24.741838 4162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:24.742082 4162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:24.743242 4162 hybrid_clock.cc:648] HybridClock initialized: now 1786183404743228 us; error 56 us; skew 500 ppm
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:24.745850 4162 init.cc:377] Logged in from keytab as oryx/127.24.162.131@KRBTEST.COM (short username oryx)
I20260808 10:03:24.746306 4162 webserver.cc:460] Webserver started at http://127.24.162.131:40485/ using document root <none> and password file <none>
I20260808 10:03:24.746546 4162 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:24.746618 4162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:24.746771 4162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:24.747927 4162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "3f38771f88ffb8739545a41a248bfa39"
server_key_iv: "7a74a0500ca3b2b7498c2eea0182cd4a"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.748282 4162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "3f38771f88ffb8739545a41a248bfa39"
server_key_iv: "7a74a0500ca3b2b7498c2eea0182cd4a"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.749607 4162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:24.750308 4186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.750501 4162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:24.750610 4162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
format_stamp: "Formatted at 2026-08-08 10:03:24 on dist-test-slave-nfb5"
server_key: "3f38771f88ffb8739545a41a248bfa39"
server_key_iv: "7a74a0500ca3b2b7498c2eea0182cd4a"
server_key_version: "encryptionkey@0"
I20260808 10:03:24.750684 4162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:24.784504 4162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:24.787037 4162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:24.787205 4162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:24.787758 4162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:24.787837 4162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.787884 4162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:24.787912 4162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:24.797207 4162 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:45769
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:24.797887 4162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:24.798135 4349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:45769 every 8 connection(s)
I20260808 10:03:24.801600 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.788344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:35573 (local address 127.24.162.190:37897)
0808 10:03:24.788398 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.788400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.788611 (+ 211us) server_negotiation.cc:407] Connection header received
0808 10:03:24.789156 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.789158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.789174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.789253 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.790347 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.791161 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.794758 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.795161 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.795227 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.798879 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.798880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.798881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.798889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.800410 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.800834 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.800835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.800835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.800888 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.801075 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.801076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.801076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.801176 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.801221 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.801476 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.801536 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:03:24.801910 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.788330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.131:35573)
0808 10:03:24.788488 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.788526 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.789040 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.789449 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.789454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.789884 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.790224 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.790232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.792988 (+ 2756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.792989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.794603 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.794606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.795408 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.795409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.795583 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.796156 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.796167 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.798753 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.800495 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.800502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.800504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.800761 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.800931 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.800931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.800932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.801024 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.801246 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.801251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.801368 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.801460 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.801582 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260808 10:03:24.802486 4350 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37897
I20260808 10:03:24.802575 4350 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:24.802757 4350 heartbeater.cc:511] Master 127.24.162.190:37897 requested a full tablet report, sending...
I20260808 10:03:24.803179 3670 ts_manager.cc:194] Registered new tserver with Master: 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:24.803754 3670 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:35573
I20260808 10:03:24.805533 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 4162
I20260808 10:03:24.805634 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:24.805840 25226 external_mini_cluster.cc:1428] Setting key 15125d35a2d59259bf6f8e300ea1d013
I20260808 10:03:24.806789 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:03:24.809839 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.803702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:45747 (local address 127.24.162.190:37897)
0808 10:03:24.803744 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.803746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.803834 (+ 88us) server_negotiation.cc:407] Connection header received
0808 10:03:24.803881 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.803882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.803891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.803949 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.804298 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.804813 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.805606 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.805836 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.805929 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.806940 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.806941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.806941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.806947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.808739 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.809053 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.809054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.809055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.809121 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.809364 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.809365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.809365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.809498 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.809556 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.809717 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.809776 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:03:24.809954 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.803602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.24.162.131:45747)
0808 10:03:24.803749 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.803761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.803830 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.803955 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.803956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.804139 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:24.804211 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.804213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.804919 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.804920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.805511 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.805513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.805933 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.805934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.806011 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.806528 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:24.806533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:24.806808 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:24.808821 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.808822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.808822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.808978 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.809203 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:24.809204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:24.809204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:24.809279 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:24.809566 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:24.809567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:24.809618 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.809701 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.809823 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:03:24.829370 3740 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:24.824098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.24.162.190:37897)
0808 10:03:24.824161 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.824163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.824172 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:24.824201 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.824202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.824210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.824265 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.824859 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.825320 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.826045 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.826230 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.826305 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.829284 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44552: BlockingRecv error: recv got EOF from 127.0.0.1:44552 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 10:03:24 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:24.837442 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.830320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.24.162.190:37897)
0808 10:03:24.830383 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.830386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.830467 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:03:24.830561 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.830562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.830570 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.830615 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:24.831152 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.831602 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.832253 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.832449 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.832534 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.835102 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:24.835103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:24.835103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:24.835109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:24.836483 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.836842 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.836843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.836843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.836895 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:24.837050 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:24.837050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:24.837050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:24.837143 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:24.837215 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.837300 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.837355 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:03:24.838955 3670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:24.840543 3670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:24.847225 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.843068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:45769 (local address 127.0.0.1:55230)
0808 10:03:24.843186 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.843198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.843248 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.843928 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.843932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.843935 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:24.844011 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.844014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.844875 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.844876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.846003 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.846005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.846890 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.846891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.847013 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.847028 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.847104 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.847152 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260808 10:03:24.847509 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.843185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.24.162.131:45769)
0808 10:03:24.843393 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.843398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.843407 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:24.843448 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.843450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.843462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.843561 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:24.844086 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.844764 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.846092 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.846794 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.846894 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.846987 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.847135 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.847180 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260808 10:03:24.847694 4381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.843271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:36481 (local address 127.0.0.1:40656)
0808 10:03:24.843643 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.843653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.843785 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.844075 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.844076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.844078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:24.844115 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.844117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.845499 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.845500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.846409 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.846411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.847481 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.847481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.847534 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.847538 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.847584 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.847622 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":73,"threads_started":1}
I20260808 10:03:24.847986 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.843414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.24.162.130:36481)
0808 10:03:24.843617 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.843622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.843771 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:03:24.843886 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.843891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.843911 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.844000 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:24.844203 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.845388 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.846495 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.847337 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.847431 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.847484 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.847846 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.847890 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":129,"threads_started":1}
I20260808 10:03:24.848235 4259 tablet_service.cc:1467] Processing CreateTablet for tablet a2dd59d4c5644605b712cdf32d04950b (DEFAULT_TABLE table=test-table [id=8475ba0317bc405ea02748a455ae44ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:24.848636 4259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dd59d4c5644605b712cdf32d04950b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:24.848681 4377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.842820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44559 (local address 127.0.0.1:60774)
0808 10:03:24.842979 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.843019 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.843099 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.843968 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.843970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.843972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:24.844034 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.844036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.846481 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.846481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.847385 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.847388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.848461 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.848462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.848529 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.848533 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.848585 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.848628 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260808 10:03:24.848621 4072 tablet_service.cc:1467] Processing CreateTablet for tablet a2dd59d4c5644605b712cdf32d04950b (DEFAULT_TABLE table=test-table [id=8475ba0317bc405ea02748a455ae44ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:24.848942 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.842974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.24.162.129:44559)
0808 10:03:24.843684 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.843689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.843698 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:24.843738 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.843739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.843755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.843963 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:24.844132 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.845380 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.847457 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.848269 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.848362 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.848418 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.848810 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.848857 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":86,"threads_started":1}
I20260808 10:03:24.848973 4072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dd59d4c5644605b712cdf32d04950b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:24.849465 3856 tablet_service.cc:1467] Processing CreateTablet for tablet a2dd59d4c5644605b712cdf32d04950b (DEFAULT_TABLE table=test-table [id=8475ba0317bc405ea02748a455ae44ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:24.849713 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dd59d4c5644605b712cdf32d04950b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:24.852861 4383 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Bootstrap starting.
I20260808 10:03:24.853185 4385 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Bootstrap starting.
I20260808 10:03:24.853807 4385 tablet_bootstrap.cc:654] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:24.853807 4383 tablet_bootstrap.cc:654] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:24.854164 4385 log.cc:824] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:24.854173 4383 log.cc:824] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:24.854519 4384 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Bootstrap starting.
I20260808 10:03:24.855053 4383 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: No bootstrap required, opened a new log
I20260808 10:03:24.855141 4383 ts_tablet_manager.cc:1397] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:24.855294 4385 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: No bootstrap required, opened a new log
I20260808 10:03:24.855369 4385 ts_tablet_manager.cc:1397] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:24.855453 4384 tablet_bootstrap.cc:654] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:24.855741 4384 log.cc:824] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:24.856443 4384 tablet_bootstrap.cc:492] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: No bootstrap required, opened a new log
I20260808 10:03:24.856524 4384 ts_tablet_manager.cc:1397] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:24.857462 4383 raft_consensus.cc:348] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.857702 4383 raft_consensus.cc:374] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:24.857563 4385 raft_consensus.cc:348] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.857762 4383 raft_consensus.cc:729] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Initialized, Role: FOLLOWER
I20260808 10:03:24.857789 4385 raft_consensus.cc:374] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:24.857829 4385 raft_consensus.cc:729] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Initialized, Role: FOLLOWER
I20260808 10:03:24.857904 4383 consensus_queue.cc:260] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.857942 4385 consensus_queue.cc:260] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.858229 4385 ts_tablet_manager.cc:1428] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:24.858222 4383 ts_tablet_manager.cc:1428] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:24.858372 4152 heartbeater.cc:503] Master 127.24.162.190:37897 was elected leader, sending a full tablet report...
I20260808 10:03:24.858647 3961 heartbeater.cc:503] Master 127.24.162.190:37897 was elected leader, sending a full tablet report...
I20260808 10:03:24.859615 4384 raft_consensus.cc:348] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.859779 4384 raft_consensus.cc:374] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:24.859838 4384 raft_consensus.cc:729] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de0e30fa57d43e9a53d46822bb36fbd, State: Initialized, Role: FOLLOWER
I20260808 10:03:24.859952 4384 consensus_queue.cc:260] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.860225 4384 ts_tablet_manager.cc:1428] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 10:03:24.860306 4350 heartbeater.cc:503] Master 127.24.162.190:37897 was elected leader, sending a full tablet report...
W20260808 10:03:24.868305 4153 tablet.cc:2369] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:03:24.950878 3962 tablet.cc:2369] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:24.951490 4394 raft_consensus.cc:482] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:24.951591 4394 raft_consensus.cc:504] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.951936 4394 leader_election.cc:288] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:24.955119 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.952156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44559 (local address 127.24.162.130:34415)
0808 10:03:24.952222 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.952239 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.952301 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.952474 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.952476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.952479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:24.952576 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.952579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.953366 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.953367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.954220 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.954223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.954786 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.954880 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.954885 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.954972 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.955016 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 10:03:24.955152 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.952258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:34415 (local address 127.24.162.129:44559)
0808 10:03:24.952309 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.952311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.952323 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:24.952377 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.952378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.952388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.952437 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:24.952647 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.953240 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954299 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.954708 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954764 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.954771 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.954999 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.955040 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:03:24.955255 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.952284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:45769 (local address 127.24.162.130:57489)
0808 10:03:24.952501 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:24.952510 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:24.952560 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:24.952706 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:24.952707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:24.952709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:24.952793 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.952794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.953420 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.953420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.954327 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:24.954330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.954968 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:24.955053 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.955057 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:24.955117 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:03:24.955154 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260808 10:03:24.955272 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.952403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:57489 (local address 127.24.162.131:45769)
0808 10:03:24.952478 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.952480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.952573 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:03:24.952618 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.952619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.952630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.952683 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:24.952865 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.953305 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954436 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.954843 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.954943 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.954953 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.955143 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.955189 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:03:24.955421 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dd59d4c5644605b712cdf32d04950b" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59" is_pre_election: true
I20260808 10:03:24.955574 3892 raft_consensus.cc:2454] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:24.955804 4000 leader_election.cc:302] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:24.955796 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dd59d4c5644605b712cdf32d04950b" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" is_pre_election: true
I20260808 10:03:24.955931 4394 raft_consensus.cc:2781] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:24.955964 4286 raft_consensus.cc:2454] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:24.955991 4394 raft_consensus.cc:482] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:24.956017 4394 raft_consensus.cc:3037] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:24.956997 4394 raft_consensus.cc:504] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.957141 4394 leader_election.cc:288] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Requested vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:24.957290 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dd59d4c5644605b712cdf32d04950b" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59"
I20260808 10:03:24.957324 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dd59d4c5644605b712cdf32d04950b" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
I20260808 10:03:24.957386 4286 raft_consensus.cc:3037] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:24.957386 3892 raft_consensus.cc:3037] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:24.958287 3892 raft_consensus.cc:2454] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:24.958289 4286 raft_consensus.cc:2454] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:24.958477 4000 leader_election.cc:302] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de0e30fa57d43e9a53d46822bb36fbd, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:24.958586 4394 raft_consensus.cc:2781] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:24.958752 4394 raft_consensus.cc:686] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Becoming Leader. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Running, Role: LEADER
I20260808 10:03:24.958889 4394 consensus_queue.cc:237] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } }
I20260808 10:03:24.959769 3670 catalog_manager.cc:5495] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } health_report { overall_health: HEALTHY } } }
I20260808 10:03:24.991889 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:24.989401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.24.162.130:36481)
0808 10:03:24.989459 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:24.989462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:24.989471 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:24.989511 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:24.989512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:24.989533 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:24.989621 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:24.989784 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.990267 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.991066 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:24.991278 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:24.991361 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:24.991546 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:24.991655 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:24.991750 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:24.991775 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 10:03:24.991795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:03:24.993817 4286 raft_consensus.cc:1261] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:24.993801 3892 raft_consensus.cc:1261] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:24.994199 4394 consensus_queue.cc:1035] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:24.994377 4399 consensus_queue.cc:1035] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:24.997154 4407 mvcc.cc:205] Tried to move back new op lower bound from 7316207226851340288 to 7316207226712276992. Current Snapshot: MvccSnapshot[applied={T|T < 7316207226851340288}]
I20260808 10:03:24.997469 4408 mvcc.cc:205] Tried to move back new op lower bound from 7316207226851340288 to 7316207226712276992. Current Snapshot: MvccSnapshot[applied={T|T < 7316207226851340288}]
I20260808 10:03:24.999220 3670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:24.999341 3670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.000206 3670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Sending DeleteTablet for 3 replicas of tablet a2dd59d4c5644605b712cdf32d04950b
I20260808 10:03:25.000525 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dd59d4c5644605b712cdf32d04950b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:24 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260808 10:03:25.000592 4259 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dd59d4c5644605b712cdf32d04950b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:24 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:55230
I20260808 10:03:25.000634 4072 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dd59d4c5644605b712cdf32d04950b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:24 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:40656
I20260808 10:03:25.000859 4422 tablet_replica.cc:330] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: stopping tablet replica
I20260808 10:03:25.000967 4422 raft_consensus.cc:2229] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.000946 3670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:03:25.001080 4422 raft_consensus.cc:2258] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:03:25.001116 3670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:25.001004 4423 tablet_replica.cc:330] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: stopping tablet replica
I20260808 10:03:25.001415 4423 raft_consensus.cc:2229] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:25.001570 4422 ts_tablet_manager.cc:1905] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.001597 4423 raft_consensus.cc:2258] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.002205 4423 ts_tablet_manager.cc:1905] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.002758 3856 tablet_service.cc:1467] Processing CreateTablet for tablet a195fec986504a72b0a3b12210631ac4 (DEFAULT_TABLE table=test-table [id=943a9a8cc099423f885c21387c696077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.003000 4072 tablet_service.cc:1467] Processing CreateTablet for tablet a195fec986504a72b0a3b12210631ac4 (DEFAULT_TABLE table=test-table [id=943a9a8cc099423f885c21387c696077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.003106 4072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a195fec986504a72b0a3b12210631ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.003427 4422 ts_tablet_manager.cc:1918] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.003510 4422 log.cc:1196] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/a2dd59d4c5644605b712cdf32d04950b
I20260808 10:03:25.003796 4422 ts_tablet_manager.cc:1939] T a2dd59d4c5644605b712cdf32d04950b P 33fec224d4174ea4a82092f759fd8a59: Deleting consensus metadata
I20260808 10:03:25.004217 3651 catalog_manager.cc:4841] TS 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559): tablet a2dd59d4c5644605b712cdf32d04950b (table test-table [id=8475ba0317bc405ea02748a455ae44ea]) successfully deleted
I20260808 10:03:25.004390 4385 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Bootstrap starting.
I20260808 10:03:25.002892 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a195fec986504a72b0a3b12210631ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.004626 4423 ts_tablet_manager.cc:1918] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.004781 4423 log.cc:1196] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/a2dd59d4c5644605b712cdf32d04950b
I20260808 10:03:25.004833 4385 tablet_bootstrap.cc:654] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.005131 4423 ts_tablet_manager.cc:1939] T a2dd59d4c5644605b712cdf32d04950b P e2e6abf8c4b0472abfbbe86128b5c540: Deleting consensus metadata
I20260808 10:03:25.005427 4385 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: No bootstrap required, opened a new log
I20260808 10:03:25.005486 4385 ts_tablet_manager.cc:1397] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:25.005620 4385 raft_consensus.cc:348] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.005667 3651 catalog_manager.cc:4841] TS e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481): tablet a2dd59d4c5644605b712cdf32d04950b (table test-table [id=8475ba0317bc405ea02748a455ae44ea]) successfully deleted
I20260808 10:03:25.005692 4385 raft_consensus.cc:374] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.005714 4385 raft_consensus.cc:729] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.005766 4385 consensus_queue.cc:260] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.005864 4385 ts_tablet_manager.cc:1428] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.005957 4383 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Bootstrap starting.
I20260808 10:03:25.006330 4383 tablet_bootstrap.cc:654] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.006999 4259 tablet_service.cc:1467] Processing CreateTablet for tablet a195fec986504a72b0a3b12210631ac4 (DEFAULT_TABLE table=test-table [id=943a9a8cc099423f885c21387c696077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.007102 4383 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: No bootstrap required, opened a new log
I20260808 10:03:25.007129 4259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a195fec986504a72b0a3b12210631ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.007146 4383 ts_tablet_manager.cc:1397] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:25.007295 4424 tablet_replica.cc:330] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: stopping tablet replica
I20260808 10:03:25.007280 4383 raft_consensus.cc:348] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.007341 4383 raft_consensus.cc:374] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.007364 4383 raft_consensus.cc:729] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.007376 4424 raft_consensus.cc:2229] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.007409 4383 consensus_queue.cc:260] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.007473 4424 raft_consensus.cc:2258] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.007692 4383 ts_tablet_manager.cc:1428] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:25.007933 4424 ts_tablet_manager.cc:1905] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.008553 4384 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Bootstrap starting.
I20260808 10:03:25.008957 4384 tablet_bootstrap.cc:654] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.009343 4424 ts_tablet_manager.cc:1918] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.009423 4424 log.cc:1196] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/a2dd59d4c5644605b712cdf32d04950b
I20260808 10:03:25.009703 4424 ts_tablet_manager.cc:1939] T a2dd59d4c5644605b712cdf32d04950b P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting consensus metadata
I20260808 10:03:25.009788 4384 tablet_bootstrap.cc:492] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: No bootstrap required, opened a new log
I20260808 10:03:25.009840 4384 ts_tablet_manager.cc:1397] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:25.009979 4384 raft_consensus.cc:348] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.010054 4384 raft_consensus.cc:374] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.010075 4384 raft_consensus.cc:729] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de0e30fa57d43e9a53d46822bb36fbd, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.010121 4384 consensus_queue.cc:260] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.010216 4384 ts_tablet_manager.cc:1428] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.010335 3651 catalog_manager.cc:4841] TS 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769): tablet a2dd59d4c5644605b712cdf32d04950b (table test-table [id=8475ba0317bc405ea02748a455ae44ea]) successfully deleted
W20260808 10:03:25.048698 4353 tablet.cc:2369] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:03:25.093454 4411 raft_consensus.cc:482] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:25.093556 4411 raft_consensus.cc:504] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.093727 4411 leader_election.cc:288] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.093991 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a195fec986504a72b0a3b12210631ac4" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59" is_pre_election: true
I20260808 10:03:25.094029 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a195fec986504a72b0a3b12210631ac4" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" is_pre_election: true
I20260808 10:03:25.094118 3892 raft_consensus.cc:2454] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:25.094146 4286 raft_consensus.cc:2454] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:25.094292 4000 leader_election.cc:302] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de0e30fa57d43e9a53d46822bb36fbd, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:25.094430 4411 raft_consensus.cc:2781] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:25.094475 4411 raft_consensus.cc:482] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:25.094494 4411 raft_consensus.cc:3037] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.095314 4411 raft_consensus.cc:504] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.095441 4411 leader_election.cc:288] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Requested vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.095615 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a195fec986504a72b0a3b12210631ac4" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59"
I20260808 10:03:25.095629 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a195fec986504a72b0a3b12210631ac4" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
I20260808 10:03:25.095705 3892 raft_consensus.cc:3037] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.095705 4286 raft_consensus.cc:3037] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.096483 4286 raft_consensus.cc:2454] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:25.096495 3892 raft_consensus.cc:2454] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:25.096665 4000 leader_election.cc:302] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:25.096767 4411 raft_consensus.cc:2781] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:25.096837 4411 raft_consensus.cc:686] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Becoming Leader. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Running, Role: LEADER
I20260808 10:03:25.096897 4411 consensus_queue.cc:237] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.097525 3669 catalog_manager.cc:5495] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183404, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.190@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:25.134655 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.127743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.24.162.190:37897)
0808 10:03:25.127807 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.127810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.127929 (+ 119us) server_negotiation.cc:407] Connection header received
0808 10:03:25.127976 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.127976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.127986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.128050 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:25.128502 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.128992 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.129729 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.129912 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.129966 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.132179 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:25.132180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:25.132181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:25.132187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:25.133652 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:25.133994 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:25.133994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:25.133995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:25.134047 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:25.134214 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:25.134215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:25.134215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:25.134308 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:25.134359 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.134517 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.134584 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:03:25.134758 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.127665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37897 (local address 127.0.0.1:44562)
0808 10:03:25.127845 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.127858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.127929 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.128068 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.128069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.128390 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:25.128440 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.128442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.129068 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.129069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.129660 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.129663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.129980 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.129981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.130021 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.130600 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:25.130605 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:25.132095 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:25.133731 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:25.133738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:25.133740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:25.133931 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:25.134105 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:25.134105 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:25.134106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:25.134161 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:25.134379 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:25.134388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:25.134439 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.134498 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.134645 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":39,"thread_start_us":120,"threads_started":1}
I20260808 10:03:25.136639 3669 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:44562:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:03:25.136884 3669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:25.138447 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 4419b63e910b4b9d881ab9e582242551 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ceb9c0b16601459ea82113fc071ec717]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:03:25.138491 4072 tablet_service.cc:1467] Processing CreateTablet for tablet 4419b63e910b4b9d881ab9e582242551 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ceb9c0b16601459ea82113fc071ec717]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:03:25.138506 4259 tablet_service.cc:1467] Processing CreateTablet for tablet 4419b63e910b4b9d881ab9e582242551 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ceb9c0b16601459ea82113fc071ec717]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:03:25.138623 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419b63e910b4b9d881ab9e582242551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.138623 4072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419b63e910b4b9d881ab9e582242551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.138674 4259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419b63e910b4b9d881ab9e582242551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.139928 4384 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: Bootstrap starting.
I20260808 10:03:25.140177 4385 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: Bootstrap starting.
I20260808 10:03:25.140204 4383 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: Bootstrap starting.
I20260808 10:03:25.140440 4384 tablet_bootstrap.cc:654] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.140507 4385 tablet_bootstrap.cc:654] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.140579 4383 tablet_bootstrap.cc:654] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.141119 4385 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: No bootstrap required, opened a new log
I20260808 10:03:25.141183 4385 ts_tablet_manager.cc:1397] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:25.141292 4383 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: No bootstrap required, opened a new log
I20260808 10:03:25.141346 4383 ts_tablet_manager.cc:1397] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:25.141311 4385 raft_consensus.cc:348] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.141405 4385 raft_consensus.cc:374] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.141435 4385 raft_consensus.cc:729] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.141489 4385 consensus_queue.cc:260] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.141503 4383 raft_consensus.cc:348] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.141578 4383 raft_consensus.cc:374] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.141583 4385 ts_tablet_manager.cc:1428] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:25.141606 4383 raft_consensus.cc:729] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.141608 4411 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.141659 4383 consensus_queue.cc:260] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.141711 4411 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.141750 4383 ts_tablet_manager.cc:1428] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:25.141906 4393 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.141984 4393 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.141997 4384 tablet_bootstrap.cc:492] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: No bootstrap required, opened a new log
I20260808 10:03:25.142041 4384 ts_tablet_manager.cc:1397] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:25.142207 4384 raft_consensus.cc:348] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.142280 4384 raft_consensus.cc:374] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.142300 4384 raft_consensus.cc:729] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de0e30fa57d43e9a53d46822bb36fbd, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.142380 4384 consensus_queue.cc:260] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.142479 4384 ts_tablet_manager.cc:1428] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.142539 4395 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.142629 4395 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.169765 4393 raft_consensus.cc:482] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:25.169866 4393 raft_consensus.cc:504] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.170164 4393 leader_election.cc:288] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.173483 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.170748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:39935 (local address 127.24.162.131:45769)
0808 10:03:25.170803 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.170805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.170893 (+ 88us) server_negotiation.cc:407] Connection header received
0808 10:03:25.170983 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.170984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.170994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.171049 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:25.171211 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.171679 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.172716 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.173067 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.173154 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.173164 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.173371 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.173412 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:03:25.173497 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.170651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:45769 (local address 127.24.162.129:39935)
0808 10:03:25.170798 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.170810 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.170904 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.171055 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.171056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.171058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:25.171137 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.171139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.171777 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.171778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.172619 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.172621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.173154 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.173154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.173276 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.173281 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.173340 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.173381 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":23,"thread_start_us":75,"threads_started":1}
I20260808 10:03:25.173513 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.170372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:36481 (local address 127.24.162.129:59389)
0808 10:03:25.170605 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.170623 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.170753 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.170996 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.170997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.170999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:25.171078 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.171080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.171755 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.171756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.172628 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.172631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.173207 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.173208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.173286 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.173289 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.173364 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.173407 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":33,"thread_start_us":149,"threads_started":1}
I20260808 10:03:25.173760 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.170477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59389 (local address 127.24.162.130:36481)
0808 10:03:25.170579 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.170581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.170735 (+ 154us) server_negotiation.cc:407] Connection header received
0808 10:03:25.170881 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.170883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.170896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.170991 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:25.171180 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.171664 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.172738 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.173110 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.173207 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.173215 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.173678 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.173715 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:03:25.174129 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419b63e910b4b9d881ab9e582242551" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" is_pre_election: true
I20260808 10:03:25.174130 4099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419b63e910b4b9d881ab9e582242551" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" is_pre_election: true
I20260808 10:03:25.174239 4286 raft_consensus.cc:2454] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 0.
I20260808 10:03:25.174258 4099 raft_consensus.cc:2454] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 0.
I20260808 10:03:25.174429 3797 leader_election.cc:302] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, 6de0e30fa57d43e9a53d46822bb36fbd; no voters:
I20260808 10:03:25.174597 4393 raft_consensus.cc:2781] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:25.174645 4393 raft_consensus.cc:482] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:25.174678 4393 raft_consensus.cc:3037] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.175390 4393 raft_consensus.cc:504] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.175521 4393 leader_election.cc:288] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 election: Requested vote from peers e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.175686 4099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419b63e910b4b9d881ab9e582242551" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e6abf8c4b0472abfbbe86128b5c540"
I20260808 10:03:25.175712 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419b63e910b4b9d881ab9e582242551" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
I20260808 10:03:25.175776 4099 raft_consensus.cc:3037] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.175776 4286 raft_consensus.cc:3037] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.176514 4286 raft_consensus.cc:2454] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 1.
I20260808 10:03:25.176610 4099 raft_consensus.cc:2454] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 1.
I20260808 10:03:25.176734 3797 leader_election.cc:302] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, 6de0e30fa57d43e9a53d46822bb36fbd; no voters:
I20260808 10:03:25.176856 4393 raft_consensus.cc:2781] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:25.177019 4393 raft_consensus.cc:686] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [term 1 LEADER]: Becoming Leader. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Running, Role: LEADER
I20260808 10:03:25.177106 4393 consensus_queue.cc:237] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.177484 4446 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: TxnStatusTablet state changed. Reason: New leader 33fec224d4174ea4a82092f759fd8a59. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.177569 4446 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:03:25.177734 4448 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:03:25.177860 4448 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:03:25.177860 3669 catalog_manager.cc:5495] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:25.191491 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.188557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44559 (local address 127.0.0.1:60790)
0808 10:03:25.188634 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.188645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.188749 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.188919 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.188919 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.188921 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:03:25.188994 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.188997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.189927 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.189928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.190503 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.190505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.190989 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.190990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.191049 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.191066 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.191346 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.191354 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.191393 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.191434 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:03:25.191496 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.188614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.24.162.129:44559)
0808 10:03:25.188655 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.188657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.188750 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:03:25.188811 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.188812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.188832 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.188916 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:25.189064 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.189844 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.190587 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.190892 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.190980 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.191156 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.191277 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.191361 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.191397 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.191417 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:03:25.192654 4286 raft_consensus.cc:1261] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Refusing update from remote peer 33fec224d4174ea4a82092f759fd8a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.192739 4099 raft_consensus.cc:1261] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Refusing update from remote peer 33fec224d4174ea4a82092f759fd8a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.192938 4446 consensus_queue.cc:1035] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.193078 4393 consensus_queue.cc:1035] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.193738 4411 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: TxnStatusTablet state changed. Reason: New leader 33fec224d4174ea4a82092f759fd8a59. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.193841 4411 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.194205 4395 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: TxnStatusTablet state changed. Reason: New leader 33fec224d4174ea4a82092f759fd8a59. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.194320 4395 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.194301 4393 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.194376 4393 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:03:25.195010 4411 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.195087 4411 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P e2e6abf8c4b0472abfbbe86128b5c540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.195842 4393 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.195928 4393 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 33fec224d4174ea4a82092f759fd8a59: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:03:25.196537 4395 tablet_replica.cc:439] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } } }
I20260808 10:03:25.196625 4395 tablet_replica.cc:442] T 4419b63e910b4b9d881ab9e582242551 P 6de0e30fa57d43e9a53d46822bb36fbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:03:25.200793 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.198207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:42513 (local address 127.24.162.129:44559)
0808 10:03:25.198255 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.198257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.198330 (+ 73us) server_negotiation.cc:407] Connection header received
0808 10:03:25.198374 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.198375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.198385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.198447 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:25.198524 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.199265 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.199995 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.200295 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.200373 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.200517 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.200550 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.200605 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.200705 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.200724 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:03:25.200798 4458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.198117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44559 (local address 127.24.162.130:42513)
0808 10:03:25.198264 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.198281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.198335 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.198448 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.198449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.198450 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:03:25.198489 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.198490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.199347 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.199348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.199914 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.199916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.200381 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.200382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.200425 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.200439 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.200611 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.200612 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.200700 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.200743 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260808 10:03:25.203027 4286 raft_consensus.cc:1261] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.203063 3892 raft_consensus.cc:1261] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.203249 4394 consensus_queue.cc:1035] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.203341 4411 consensus_queue.cc:1035] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.204545 4402 mvcc.cc:205] Tried to move back new op lower bound from 7316207227710119936 to 7316207227277230080. Current Snapshot: MvccSnapshot[applied={T|T < 7316207227710119936}]
I20260808 10:03:25.211673 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.209208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45257 (local address 127.24.162.130:36481)
0808 10:03:25.209321 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.209324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.209373 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:03:25.209453 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.209454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.209463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.209502 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:25.209635 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.210087 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.210864 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.211190 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.211270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.211457 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.211491 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.211538 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.211589 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.211608 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 10:03:25.211737 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.209161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:36481 (local address 127.24.162.129:45257)
0808 10:03:25.209315 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.209327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:25.209396 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:25.209513 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:25.209514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:25.209515 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:03:25.209569 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.209571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.210174 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.210175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.210772 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:25.210774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.211284 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.211285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:25.211349 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.211372 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.211538 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.211540 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:03:25.211586 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:03:25.211628 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260808 10:03:25.220820 3670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.220914 3670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.221589 3670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Sending DeleteTablet for 3 replicas of tablet a195fec986504a72b0a3b12210631ac4
I20260808 10:03:25.221839 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet a195fec986504a72b0a3b12210631ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260808 10:03:25.221935 4422 tablet_replica.cc:330] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: stopping tablet replica
I20260808 10:03:25.221983 4422 raft_consensus.cc:2229] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.221943 4072 tablet_service.cc:1514] Processing DeleteTablet for tablet a195fec986504a72b0a3b12210631ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:40656
I20260808 10:03:25.222023 4422 raft_consensus.cc:2258] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.222045 4423 tablet_replica.cc:330] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: stopping tablet replica
I20260808 10:03:25.222093 4423 raft_consensus.cc:2229] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:25.222153 4423 raft_consensus.cc:2258] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.222240 4259 tablet_service.cc:1514] Processing DeleteTablet for tablet a195fec986504a72b0a3b12210631ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:55230
I20260808 10:03:25.222380 4424 tablet_replica.cc:330] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: stopping tablet replica
I20260808 10:03:25.222442 4424 raft_consensus.cc:2229] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.222486 4424 raft_consensus.cc:2258] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.222513 4423 ts_tablet_manager.cc:1905] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.222613 4422 ts_tablet_manager.cc:1905] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.224231 4422 ts_tablet_manager.cc:1918] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:03:25.224324 4422 log.cc:1196] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/a195fec986504a72b0a3b12210631ac4
I20260808 10:03:25.224548 4422 ts_tablet_manager.cc:1939] T a195fec986504a72b0a3b12210631ac4 P 33fec224d4174ea4a82092f759fd8a59: Deleting consensus metadata
I20260808 10:03:25.224866 4423 ts_tablet_manager.cc:1918] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:03:25.224881 3651 catalog_manager.cc:4841] TS 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559): tablet a195fec986504a72b0a3b12210631ac4 (table test-table [id=943a9a8cc099423f885c21387c696077]) successfully deleted
I20260808 10:03:25.224926 4423 log.cc:1196] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/a195fec986504a72b0a3b12210631ac4
I20260808 10:03:25.225142 4423 ts_tablet_manager.cc:1939] T a195fec986504a72b0a3b12210631ac4 P e2e6abf8c4b0472abfbbe86128b5c540: Deleting consensus metadata
I20260808 10:03:25.225452 3651 catalog_manager.cc:4841] TS e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481): tablet a195fec986504a72b0a3b12210631ac4 (table test-table [id=943a9a8cc099423f885c21387c696077]) successfully deleted
I20260808 10:03:25.227255 4424 ts_tablet_manager.cc:1905] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.229113 4424 ts_tablet_manager.cc:1918] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:03:25.229193 4424 log.cc:1196] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/a195fec986504a72b0a3b12210631ac4
I20260808 10:03:25.229417 4424 ts_tablet_manager.cc:1939] T a195fec986504a72b0a3b12210631ac4 P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting consensus metadata
I20260808 10:03:25.229760 3651 catalog_manager.cc:4841] TS 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769): tablet a195fec986504a72b0a3b12210631ac4 (table test-table [id=943a9a8cc099423f885c21387c696077]) successfully deleted
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:03:25.248620 3740 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:25.242357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.24.162.190:37897)
0808 10:03:25.242436 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.242439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.242546 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:03:25.242583 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.242584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.242598 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.242682 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:25.243331 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.244116 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.244881 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.245191 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.245268 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.248559 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44564: BlockingRecv error: recv got EOF from 127.0.0.1:44564 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM
I20260808 10:03:25.258801 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.249786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.24.162.190:37897)
0808 10:03:25.249821 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.249824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.251015 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 10:03:25.251050 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.251051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.251061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.251128 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:25.251573 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.252338 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.253034 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.253345 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.253416 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.255873 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:25.255875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:25.255875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:25.255883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:25.257584 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:25.257923 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:25.257924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:25.257925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:25.257992 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:25.258249 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:25.258250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:25.258251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:25.258403 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:25.258513 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.258597 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.258712 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:03:25.260450 3670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:25.260627 3670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:25.262161 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 972533aeacd24f37858744f0cca5544d (DEFAULT_TABLE table=test-table [id=f84104e4c23a4faea50550741ce32ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.262158 4072 tablet_service.cc:1467] Processing CreateTablet for tablet 972533aeacd24f37858744f0cca5544d (DEFAULT_TABLE table=test-table [id=f84104e4c23a4faea50550741ce32ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.262302 4072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972533aeacd24f37858744f0cca5544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.262301 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972533aeacd24f37858744f0cca5544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.263827 4385 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Bootstrap starting.
I20260808 10:03:25.263916 4259 tablet_service.cc:1467] Processing CreateTablet for tablet 972533aeacd24f37858744f0cca5544d (DEFAULT_TABLE table=test-table [id=f84104e4c23a4faea50550741ce32ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.264056 4259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972533aeacd24f37858744f0cca5544d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.264200 4385 tablet_bootstrap.cc:654] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.264498 4383 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Bootstrap starting.
I20260808 10:03:25.264791 4383 tablet_bootstrap.cc:654] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.265198 4384 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Bootstrap starting.
I20260808 10:03:25.265329 4385 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: No bootstrap required, opened a new log
I20260808 10:03:25.265374 4385 ts_tablet_manager.cc:1397] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:25.265486 4385 raft_consensus.cc:348] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.265555 4385 raft_consensus.cc:374] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.265574 4385 raft_consensus.cc:729] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.265628 4385 consensus_queue.cc:260] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.265713 4385 ts_tablet_manager.cc:1428] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.265887 4384 tablet_bootstrap.cc:654] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.266153 4383 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: No bootstrap required, opened a new log
I20260808 10:03:25.266206 4383 ts_tablet_manager.cc:1397] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:03:25.266358 4383 raft_consensus.cc:348] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.266420 4383 raft_consensus.cc:374] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.266440 4383 raft_consensus.cc:729] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.266481 4383 consensus_queue.cc:260] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.266582 4383 ts_tablet_manager.cc:1428] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.266701 4384 tablet_bootstrap.cc:492] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: No bootstrap required, opened a new log
I20260808 10:03:25.266772 4384 ts_tablet_manager.cc:1397] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:25.266973 4384 raft_consensus.cc:348] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.267071 4384 raft_consensus.cc:374] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.267097 4384 raft_consensus.cc:729] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de0e30fa57d43e9a53d46822bb36fbd, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.267153 4384 consensus_queue.cc:260] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.267300 4384 ts_tablet_manager.cc:1428] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.296171 4459 raft_consensus.cc:482] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:25.296262 4459 raft_consensus.cc:504] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.296427 4459 leader_election.cc:288] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.296633 4099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972533aeacd24f37858744f0cca5544d" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" is_pre_election: true
I20260808 10:03:25.296675 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972533aeacd24f37858744f0cca5544d" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" is_pre_election: true
I20260808 10:03:25.296749 4099 raft_consensus.cc:2454] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 0.
I20260808 10:03:25.296756 4286 raft_consensus.cc:2454] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 0.
I20260808 10:03:25.296921 3797 leader_election.cc:302] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, 6de0e30fa57d43e9a53d46822bb36fbd; no voters:
I20260808 10:03:25.297061 4459 raft_consensus.cc:2781] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:25.297116 4459 raft_consensus.cc:482] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:25.297145 4459 raft_consensus.cc:3037] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.298290 4459 raft_consensus.cc:504] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.298427 4459 leader_election.cc:288] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 election: Requested vote from peers e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.298671 4099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972533aeacd24f37858744f0cca5544d" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e6abf8c4b0472abfbbe86128b5c540"
I20260808 10:03:25.298760 4099 raft_consensus.cc:3037] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.298679 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972533aeacd24f37858744f0cca5544d" candidate_uuid: "33fec224d4174ea4a82092f759fd8a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
I20260808 10:03:25.298808 4286 raft_consensus.cc:3037] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.299511 4286 raft_consensus.cc:2454] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 1.
I20260808 10:03:25.299655 4099 raft_consensus.cc:2454] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fec224d4174ea4a82092f759fd8a59 in term 1.
I20260808 10:03:25.299705 3797 leader_election.cc:302] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, 6de0e30fa57d43e9a53d46822bb36fbd; no voters:
I20260808 10:03:25.299808 4459 raft_consensus.cc:2781] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:25.299865 4459 raft_consensus.cc:686] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 1 LEADER]: Becoming Leader. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Running, Role: LEADER
I20260808 10:03:25.299934 4459 consensus_queue.cc:237] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.300532 3670 catalog_manager.cc:5495] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "33fec224d4174ea4a82092f759fd8a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:25.306845 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.304406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.24.162.129:44559)
0808 10:03:25.304481 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.304483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.304494 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:25.304520 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.304521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.304533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.304607 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:25.304710 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.305392 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.306132 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.306420 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.306494 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.306624 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.306659 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.306714 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.306761 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.306782 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:03:25.307611 4286 raft_consensus.cc:1261] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Refusing update from remote peer 33fec224d4174ea4a82092f759fd8a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.307654 4099 raft_consensus.cc:1261] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Refusing update from remote peer 33fec224d4174ea4a82092f759fd8a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.307857 4459 consensus_queue.cc:1035] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.307948 4446 consensus_queue.cc:1035] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.311087 3670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.311183 3670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.311794 3670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Sending DeleteTablet for 3 replicas of tablet 972533aeacd24f37858744f0cca5544d
I20260808 10:03:25.312114 4259 tablet_service.cc:1514] Processing DeleteTablet for tablet 972533aeacd24f37858744f0cca5544d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:55230
I20260808 10:03:25.312219 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 972533aeacd24f37858744f0cca5544d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260808 10:03:25.312353 4422 tablet_replica.cc:330] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: stopping tablet replica
I20260808 10:03:25.312350 4072 tablet_service.cc:1514] Processing DeleteTablet for tablet 972533aeacd24f37858744f0cca5544d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:40656
I20260808 10:03:25.312415 4422 raft_consensus.cc:2229] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:25.312435 4424 tablet_replica.cc:330] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: stopping tablet replica
I20260808 10:03:25.312484 4424 raft_consensus.cc:2229] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.312502 4422 raft_consensus.cc:2258] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.312533 4424 raft_consensus.cc:2258] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.312611 4423 tablet_replica.cc:330] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: stopping tablet replica
I20260808 10:03:25.312691 4423 raft_consensus.cc:2229] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.312741 4423 raft_consensus.cc:2258] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.313048 4423 ts_tablet_manager.cc:1905] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.313067 3670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:03:25.313180 4422 ts_tablet_manager.cc:1905] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:03:25.313220 3670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:25.314553 4422 ts_tablet_manager.cc:1918] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.314561 4423 ts_tablet_manager.cc:1918] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.314584 4424 ts_tablet_manager.cc:1905] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.314610 4422 log.cc:1196] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/972533aeacd24f37858744f0cca5544d
I20260808 10:03:25.314613 4423 log.cc:1196] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/972533aeacd24f37858744f0cca5544d
I20260808 10:03:25.314845 4423 ts_tablet_manager.cc:1939] T 972533aeacd24f37858744f0cca5544d P e2e6abf8c4b0472abfbbe86128b5c540: Deleting consensus metadata
I20260808 10:03:25.314850 4422 ts_tablet_manager.cc:1939] T 972533aeacd24f37858744f0cca5544d P 33fec224d4174ea4a82092f759fd8a59: Deleting consensus metadata
I20260808 10:03:25.315192 3651 catalog_manager.cc:4841] TS e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481): tablet 972533aeacd24f37858744f0cca5544d (table test-table [id=f84104e4c23a4faea50550741ce32ff2]) successfully deleted
I20260808 10:03:25.315256 3651 catalog_manager.cc:4841] TS 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559): tablet 972533aeacd24f37858744f0cca5544d (table test-table [id=f84104e4c23a4faea50550741ce32ff2]) successfully deleted
I20260808 10:03:25.316087 4072 tablet_service.cc:1467] Processing CreateTablet for tablet 172bf812c8a042b885789141addf88da (DEFAULT_TABLE table=test-table [id=62e7843bf89b4de790e9783b888deb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.316123 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 172bf812c8a042b885789141addf88da (DEFAULT_TABLE table=test-table [id=62e7843bf89b4de790e9783b888deb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.316193 4072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172bf812c8a042b885789141addf88da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.316226 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172bf812c8a042b885789141addf88da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.316224 4424 ts_tablet_manager.cc:1918] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:25.316272 4424 log.cc:1196] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/972533aeacd24f37858744f0cca5544d
I20260808 10:03:25.316478 4424 ts_tablet_manager.cc:1939] T 972533aeacd24f37858744f0cca5544d P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting consensus metadata
I20260808 10:03:25.317349 4383 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: Bootstrap starting.
I20260808 10:03:25.317368 4385 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Bootstrap starting.
I20260808 10:03:25.317653 4385 tablet_bootstrap.cc:654] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.317715 4383 tablet_bootstrap.cc:654] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.318070 4259 tablet_service.cc:1467] Processing CreateTablet for tablet 172bf812c8a042b885789141addf88da (DEFAULT_TABLE table=test-table [id=62e7843bf89b4de790e9783b888deb7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:25.318174 4259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172bf812c8a042b885789141addf88da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:25.318249 4383 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: No bootstrap required, opened a new log
I20260808 10:03:25.318282 4383 ts_tablet_manager.cc:1397] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:25.318387 4383 raft_consensus.cc:348] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.318430 4383 raft_consensus.cc:374] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.318444 4383 raft_consensus.cc:729] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fec224d4174ea4a82092f759fd8a59, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.318504 4383 consensus_queue.cc:260] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.318627 4383 ts_tablet_manager.cc:1428] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.319157 4385 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: No bootstrap required, opened a new log
I20260808 10:03:25.319195 4385 ts_tablet_manager.cc:1397] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:03:25.319280 4385 raft_consensus.cc:348] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.319355 4385 raft_consensus.cc:374] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.319377 4385 raft_consensus.cc:729] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.319368 4384 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Bootstrap starting.
I20260808 10:03:25.319419 4385 consensus_queue.cc:260] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.319505 4385 ts_tablet_manager.cc:1428] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.319633 3651 catalog_manager.cc:4841] TS 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769): tablet 972533aeacd24f37858744f0cca5544d (table test-table [id=f84104e4c23a4faea50550741ce32ff2]) successfully deleted
I20260808 10:03:25.319741 4384 tablet_bootstrap.cc:654] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:25.320370 4384 tablet_bootstrap.cc:492] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: No bootstrap required, opened a new log
I20260808 10:03:25.320417 4384 ts_tablet_manager.cc:1397] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:25.320561 4384 raft_consensus.cc:348] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.320650 4384 raft_consensus.cc:374] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:25.320681 4384 raft_consensus.cc:729] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de0e30fa57d43e9a53d46822bb36fbd, State: Initialized, Role: FOLLOWER
I20260808 10:03:25.320724 4384 consensus_queue.cc:260] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.320839 4384 ts_tablet_manager.cc:1428] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:25.334012 4411 raft_consensus.cc:482] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:25.334105 4411 raft_consensus.cc:504] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.334276 4411 leader_election.cc:288] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.334501 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172bf812c8a042b885789141addf88da" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" is_pre_election: true
I20260808 10:03:25.334457 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172bf812c8a042b885789141addf88da" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59" is_pre_election: true
I20260808 10:03:25.334597 4286 raft_consensus.cc:2454] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:25.334599 3892 raft_consensus.cc:2454] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 0.
I20260808 10:03:25.334784 4000 leader_election.cc:302] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de0e30fa57d43e9a53d46822bb36fbd, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:25.334896 4411 raft_consensus.cc:2781] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:25.334980 4411 raft_consensus.cc:482] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:25.335011 4411 raft_consensus.cc:3037] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.335608 4411 raft_consensus.cc:504] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.335738 4411 leader_election.cc:288] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Requested vote from peers 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559), 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769)
I20260808 10:03:25.335918 4286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172bf812c8a042b885789141addf88da" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de0e30fa57d43e9a53d46822bb36fbd"
I20260808 10:03:25.335924 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172bf812c8a042b885789141addf88da" candidate_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33fec224d4174ea4a82092f759fd8a59"
I20260808 10:03:25.335995 4286 raft_consensus.cc:3037] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.336016 3892 raft_consensus.cc:3037] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:25.336601 3892 raft_consensus.cc:2454] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:25.336661 4286 raft_consensus.cc:2454] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2e6abf8c4b0472abfbbe86128b5c540 in term 1.
I20260808 10:03:25.336786 4000 leader_election.cc:302] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33fec224d4174ea4a82092f759fd8a59, e2e6abf8c4b0472abfbbe86128b5c540; no voters:
I20260808 10:03:25.336946 4411 raft_consensus.cc:2781] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:25.337016 4411 raft_consensus.cc:686] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Becoming Leader. State: Replica: e2e6abf8c4b0472abfbbe86128b5c540, State: Running, Role: LEADER
I20260808 10:03:25.337056 4411 consensus_queue.cc:237] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } }
I20260808 10:03:25.337589 3670 catalog_manager.cc:5495] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2e6abf8c4b0472abfbbe86128b5c540" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:25.345732 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.343387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.24.162.130:36481)
0808 10:03:25.343432 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:25.343434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:25.343445 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:25.343471 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:25.343471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:25.343485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:25.343554 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:25.343637 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.344365 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.345045 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:25.345349 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:25.345423 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:25.345518 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:25.345551 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:25.345600 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:25.345644 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 10:03:25.345663 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:03:25.348138 4286 raft_consensus.cc:1261] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.348127 3892 raft_consensus.cc:1261] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Refusing update from remote peer e2e6abf8c4b0472abfbbe86128b5c540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:25.348366 4411 consensus_queue.cc:1035] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de0e30fa57d43e9a53d46822bb36fbd" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.348485 4394 consensus_queue.cc:1035] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fec224d4174ea4a82092f759fd8a59" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:25.360029 3670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.360138 3670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:25.360790 3670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f16fddc91594e47a4d56f703dd6654f: Sending DeleteTablet for 3 replicas of tablet 172bf812c8a042b885789141addf88da
I20260808 10:03:25.361084 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 172bf812c8a042b885789141addf88da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260808 10:03:25.361085 4259 tablet_service.cc:1514] Processing DeleteTablet for tablet 172bf812c8a042b885789141addf88da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:55230
I20260808 10:03:25.361173 4422 tablet_replica.cc:330] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59: stopping tablet replica
I20260808 10:03:25.361173 4424 tablet_replica.cc:330] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: stopping tablet replica
I20260808 10:03:25.361219 4422 raft_consensus.cc:2229] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.361227 4424 raft_consensus.cc:2229] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:25.361258 4422 raft_consensus.cc:2258] T 172bf812c8a042b885789141addf88da P 33fec224d4174ea4a82092f759fd8a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.361258 4424 raft_consensus.cc:2258] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.361383 4072 tablet_service.cc:1514] Processing DeleteTablet for tablet 172bf812c8a042b885789141addf88da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:25 UTC) from {username='oryx', principal='oryx/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:40656
I20260808 10:03:25.361462 4423 tablet_replica.cc:330] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: stopping tablet replica
I20260808 10:03:25.361510 4423 raft_consensus.cc:2229] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:03:25.361584 4423 raft_consensus.cc:2258] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:25.361616 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 3772
I20260808 10:03:25.366544 4423 ts_tablet_manager.cc:1905] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.368768 4423 ts_tablet_manager.cc:1918] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:03:25.368839 4423 log.cc:1196] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/172bf812c8a042b885789141addf88da
I20260808 10:03:25.369110 4423 ts_tablet_manager.cc:1939] T 172bf812c8a042b885789141addf88da P e2e6abf8c4b0472abfbbe86128b5c540: Deleting consensus metadata
I20260808 10:03:25.369511 3651 catalog_manager.cc:4841] TS e2e6abf8c4b0472abfbbe86128b5c540 (127.24.162.130:36481): tablet 172bf812c8a042b885789141addf88da (table test-table [id=62e7843bf89b4de790e9783b888deb7a]) successfully deleted
I20260808 10:03:25.371940 4424 ts_tablet_manager.cc:1905] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:25.373898 4424 ts_tablet_manager.cc:1918] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:03:25.373965 4424 log.cc:1196] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/172bf812c8a042b885789141addf88da
I20260808 10:03:25.374181 4424 ts_tablet_manager.cc:1939] T 172bf812c8a042b885789141addf88da P 6de0e30fa57d43e9a53d46822bb36fbd: Deleting consensus metadata
I20260808 10:03:25.374511 3651 catalog_manager.cc:4841] TS 6de0e30fa57d43e9a53d46822bb36fbd (127.24.162.131:45769): tablet 172bf812c8a042b885789141addf88da (table test-table [id=62e7843bf89b4de790e9783b888deb7a]) successfully deleted
W20260808 10:03:25.379913 3651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:44559 (error 108)
I20260808 10:03:25.380184 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 3971
I20260808 10:03:25.383447 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:25.380266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:44559 (local address 127.0.0.1:60808)
0808 10:03:25.383313 (+ 3047us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:25.383380 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3023,"thread_start_us":40,"threads_started":1}
W20260808 10:03:25.386965 3651 catalog_manager.cc:4584] TS 33fec224d4174ea4a82092f759fd8a59 (127.24.162.129:44559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 172bf812c8a042b885789141addf88da: Network error: Client connection negotiation failed: client connection to 127.24.162.129:44559: connect: Connection refused (error 111)
I20260808 10:03:25.391847 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 4162
I20260808 10:03:25.399796 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 3626
2026-08-08T10:03:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2639 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[4505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[4505](info): set up 2 sockets
Aug 08 10:03:25 dist-test-slave-nfb5 krb5kdc[4505](info): commencing operation
krb5kdc: starting...
W20260808 10:03:26.617692 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:26 dist-test-slave-nfb5 krb5kdc[4505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:26.744105 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38979
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:38979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:26.829813 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:26.830008 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:26.830031 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:26.831174 4525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:26.831219 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:26.831234 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:26.831247 4525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:26.831259 4525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:26.832686 4525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:38979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:26.832881 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:26.833055 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:26.837883 4532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:26.837898 4531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:26.838061 4525 server_base.cc:1034] running on GCE node
W20260808 10:03:26.837905 4534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:26.838454 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:26.838783 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:26.839946 4525 hybrid_clock.cc:648] HybridClock initialized: now 1786183406839935 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T10:03:26Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1447 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:26 dist-test-slave-nfb5 krb5kdc[4541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:26 dist-test-slave-nfb5 krb5kdc[4541](info): set up 2 sockets
Aug 08 10:03:26 dist-test-slave-nfb5 krb5kdc[4541](info): commencing operation
krb5kdc: starting...
W20260808 10:03:28.068758 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:28.193338 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36427
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:28.275741 4559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:28.275944 4559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:28.275967 4559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:28.277118 4559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:28.277164 4559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:28.277179 4559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:28.277240 4559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:28.277262 4559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:28.278821 4559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:28.279116 4559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:28.279359 4559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:28.284257 4570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.284413 4572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.284341 4569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:28.284529 4559 server_base.cc:1034] running on GCE node
I20260808 10:03:28.284842 4559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:28.285218 4559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:28.286387 4559 hybrid_clock.cc:648] HybridClock initialized: now 1786183408286370 us; error 49 us; skew 500 ppm
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:28.293052 4559 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:28.293557 4559 webserver.cc:460] Webserver started at http://127.24.162.190:45123/ using document root <none> and password file <none>
I20260808 10:03:28.293812 4559 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:28.293884 4559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:28.294039 4559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:28.295202 4559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "b2083c8b089446b8a5004f355d4cea69"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "56243acf4a1d2d8b7a94c9daaf0c4c86"
server_key_iv: "ce05d460fa717aa959af45ae8a2bc7de"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.295564 4559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "b2083c8b089446b8a5004f355d4cea69"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "56243acf4a1d2d8b7a94c9daaf0c4c86"
server_key_iv: "ce05d460fa717aa959af45ae8a2bc7de"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.296903 4559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:28.297640 4583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.297813 4559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:28.297923 4559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "b2083c8b089446b8a5004f355d4cea69"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "56243acf4a1d2d8b7a94c9daaf0c4c86"
server_key_iv: "ce05d460fa717aa959af45ae8a2bc7de"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.297995 4559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:28.313174 4559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:28.315541 4559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:28.315656 4559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:28.319821 4559 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36427
I20260808 10:03:28.319877 4670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36427 every 8 connection(s)
I20260808 10:03:28.320240 4559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:28.321099 4675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:28.323376 4675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Bootstrap starting.
I20260808 10:03:28.323906 4675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:28.324149 4675 log.cc:824] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:28.324779 4675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: No bootstrap required, opened a new log
I20260808 10:03:28.326316 4675 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:28.326447 4675 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:28.326469 4675 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2083c8b089446b8a5004f355d4cea69, State: Initialized, Role: FOLLOWER
I20260808 10:03:28.326565 4675 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:28.326644 4675 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:28.326679 4675 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:28.326733 4675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:28.327342 4675 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:28.327466 4675 leader_election.cc:302] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2083c8b089446b8a5004f355d4cea69; no voters:
I20260808 10:03:28.327651 4675 leader_election.cc:288] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:28.327713 4678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:28.327920 4675 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:28.328138 4678 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 LEADER]: Becoming Leader. State: Replica: b2083c8b089446b8a5004f355d4cea69, State: Running, Role: LEADER
I20260808 10:03:28.328240 4678 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:28.328608 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 4559
I20260808 10:03:28.328742 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:28.328790 4683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2083c8b089446b8a5004f355d4cea69. Latest consensus state: current_term: 1 leader_uuid: "b2083c8b089446b8a5004f355d4cea69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } } }
I20260808 10:03:28.328871 4683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:28.328934 25226 external_mini_cluster.cc:1428] Setting key 7c0e10e5603707a150bee3f0852666ac
I20260808 10:03:28.328953 4680 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2083c8b089446b8a5004f355d4cea69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } } }
I20260808 10:03:28.329020 4680 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:28.329239 4696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:28.330020 4696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:28.332103 4696 catalog_manager.cc:1347] Generated new cluster ID: 3df208e23fc147f996f45b9b1d54ba30
I20260808 10:03:28.332168 4696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:28.339824 4696 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:28.340495 4696 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:28.340552 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.329484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.24.162.190:36427)
0808 10:03:28.329668 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:28.329678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:28.329714 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:03:28.329836 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:28.329843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:28.329899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:28.330056 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:28.330847 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.331990 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.333079 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.333675 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.333776 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.336591 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:28.336595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:28.336595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:28.336607 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:28.338674 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.339115 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.339119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.339119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.339234 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.339618 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.339619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.339620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.339909 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:28.340081 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:28.340181 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:03:28.340330 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 10:03:28.343607 4696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Generated new TSK 0
I20260808 10:03:28.343839 4696 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:28.373927 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:28.456823 4721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:28.457060 4721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:28.457098 4721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:28.458256 4721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:28.458328 4721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:28.458374 4721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:28.459843 4721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:28.460114 4721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:28.460364 4721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:28.465641 4732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.465620 4730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.465613 4728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:28.465937 4721 server_base.cc:1034] running on GCE node
I20260808 10:03:28.466142 4721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:28.466355 4721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:28.467494 4721 hybrid_clock.cc:648] HybridClock initialized: now 1786183408467481 us; error 30 us; skew 500 ppm
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:28.470082 4721 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:28.470530 4721 webserver.cc:460] Webserver started at http://127.24.162.129:38875/ using document root <none> and password file <none>
I20260808 10:03:28.470770 4721 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:28.470824 4721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:28.471042 4721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:28.472164 4721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "4649cb6dda12479fbc385f4003f650b2"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "87dfae21837da6fdfb2f7d5ba9e22772"
server_key_iv: "add1ea71c6d7510008f072571fe545fa"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.472543 4721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "4649cb6dda12479fbc385f4003f650b2"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "87dfae21837da6fdfb2f7d5ba9e22772"
server_key_iv: "add1ea71c6d7510008f072571fe545fa"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.474175 4721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:28.475023 4745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.475250 4721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:28.475354 4721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "4649cb6dda12479fbc385f4003f650b2"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "87dfae21837da6fdfb2f7d5ba9e22772"
server_key_iv: "add1ea71c6d7510008f072571fe545fa"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.475443 4721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:28.492815 4721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:28.495072 4721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:28.495241 4721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:28.495481 4721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:28.495810 4721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:28.495867 4721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.495903 4721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:28.495930 4721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.502066 4721 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:39975
I20260808 10:03:28.502108 4917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:39975 every 8 connection(s)
I20260808 10:03:28.502466 4721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:28.509300 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 4721
I20260808 10:03:28.509382 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:28.509627 25226 external_mini_cluster.cc:1428] Setting key adf5840ba9578cd7d105577183c80d58
I20260808 10:03:28.512413 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.503256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:49015 (local address 127.24.162.190:36427)
0808 10:03:28.503340 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:28.503343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:28.503535 (+ 192us) server_negotiation.cc:407] Connection header received
0808 10:03:28.503998 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:28.503999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:28.504014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:28.504065 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:28.505004 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.505521 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.506496 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.506695 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.506753 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.509013 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:28.509014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:28.509014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:28.509021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:28.511023 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.511438 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.511439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.511440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.511507 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.511732 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.511733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.511733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.511879 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:28.511951 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:28.512240 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:03:28.512341 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:03:28.512667 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.503215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:49015)
0808 10:03:28.503408 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:28.503444 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:28.503913 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:28.504122 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:28.504125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:28.504721 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:28.504910 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.504915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.505603 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.505604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.506412 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.506414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.506782 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.506783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.506887 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.507498 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:28.507506 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:28.508906 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:28.511116 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.511122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.511124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.511364 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.511573 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.511573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.511574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.511663 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.511946 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:28.511951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:28.512062 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:03:28.512217 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 10:03:28.512409 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260808 10:03:28.513172 4918 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36427
I20260808 10:03:28.513275 4918 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:28.513475 4918 heartbeater.cc:511] Master 127.24.162.190:36427 requested a full tablet report, sending...
I20260808 10:03:28.514050 4614 ts_manager.cc:194] Registered new tserver with Master: 4649cb6dda12479fbc385f4003f650b2 (127.24.162.129:39975)
I20260808 10:03:28.515044 4614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:49015
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:28.537628 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:28.619954 4924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:28.620159 4924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:28.620182 4924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:28.621290 4924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:28.621328 4924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:28.621353 4924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:28.622776 4924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:28.623023 4924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:28.623214 4924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:28.628171 4932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.628209 4933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.628154 4935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:28.628288 4924 server_base.cc:1034] running on GCE node
I20260808 10:03:28.628597 4924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:28.628854 4924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:28.630014 4924 hybrid_clock.cc:648] HybridClock initialized: now 1786183408629997 us; error 27 us; skew 500 ppm
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:28.632582 4924 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:28.632953 4924 webserver.cc:460] Webserver started at http://127.24.162.130:35459/ using document root <none> and password file <none>
I20260808 10:03:28.633142 4924 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:28.633183 4924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:28.633272 4924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:28.634238 4924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "c9c787ebb70047a8acb51b05d57695e8"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "96f9fb2c37bb7444a943c8f29074645b"
server_key_iv: "5a01a3e4fd887ef168317232bf94364e"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.634548 4924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "c9c787ebb70047a8acb51b05d57695e8"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "96f9fb2c37bb7444a943c8f29074645b"
server_key_iv: "5a01a3e4fd887ef168317232bf94364e"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.635994 4924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:28.636686 4947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.636860 4924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:28.636968 4924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "c9c787ebb70047a8acb51b05d57695e8"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "96f9fb2c37bb7444a943c8f29074645b"
server_key_iv: "5a01a3e4fd887ef168317232bf94364e"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.637042 4924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:28.648476 4924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:28.650750 4924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:28.650949 4924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:28.651196 4924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:28.651494 4924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:28.651548 4924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.651589 4924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:28.651616 4924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.657407 4924 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:39651
I20260808 10:03:28.657485 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:39651 every 8 connection(s)
I20260808 10:03:28.657797 4924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:28.662890 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 4924
I20260808 10:03:28.663055 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:28.663270 25226 external_mini_cluster.cc:1428] Setting key bcd3d1061d915e6e8369e2d8ba5e4e71
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:28.667817 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.658554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:48615 (local address 127.24.162.190:36427)
0808 10:03:28.658602 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:28.658605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:28.658813 (+ 208us) server_negotiation.cc:407] Connection header received
0808 10:03:28.659339 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:28.659340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:28.659350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:28.659414 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:28.660309 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.660821 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.661883 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.662119 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.662196 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.664775 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:28.664777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:28.664777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:28.664786 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:28.666285 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.666793 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.666794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.666794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.666846 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.667162 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.667163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.667163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.667272 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:28.667327 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:28.667664 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 10:03:28.667754 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:28.668081 5108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.658520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:48615)
0808 10:03:28.658682 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:28.658722 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:28.659254 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:28.659464 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:28.659468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:28.660022 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:28.660210 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.660214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.660917 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.660918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.661777 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.661780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.662210 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.662211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.662314 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.662837 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:28.662845 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:28.664675 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:28.666374 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.666381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.666383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.666707 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.666945 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.666945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.666946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.667093 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.667350 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:28.667357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:28.667522 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 10:03:28.667631 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:03:28.667806 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260808 10:03:28.668696 5105 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36427
I20260808 10:03:28.668815 5105 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:28.668992 5105 heartbeater.cc:511] Master 127.24.162.190:36427 requested a full tablet report, sending...
I20260808 10:03:28.669418 4614 ts_manager.cc:194] Registered new tserver with Master: c9c787ebb70047a8acb51b05d57695e8 (127.24.162.130:39651)
I20260808 10:03:28.669791 4614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:48615
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:28.690553 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:28.773432 5112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:28.773635 5112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:28.773658 5112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:28.774761 5112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:28.774794 5112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:28.774820 5112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:28.776288 5112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:28.776468 5112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:28.776643 5112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:28.781756 5119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.781864 5121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:28.781766 5118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:28.781883 5112 server_base.cc:1034] running on GCE node
I20260808 10:03:28.782303 5112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:28.782557 5112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:28.783725 5112 hybrid_clock.cc:648] HybridClock initialized: now 1786183408783709 us; error 46 us; skew 500 ppm
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:28.786252 5112 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:28.786720 5112 webserver.cc:460] Webserver started at http://127.24.162.131:33419/ using document root <none> and password file <none>
I20260808 10:03:28.787029 5112 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:28.787099 5112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:28.787240 5112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:28.788272 5112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "2bc78a650b83422487b2b2f6f7125a57"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "d37d421b70d79f213310b63e7313acf2"
server_key_iv: "41fb7b4a73d07dbd268278e0fd84aa71"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.788640 5112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "2bc78a650b83422487b2b2f6f7125a57"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "d37d421b70d79f213310b63e7313acf2"
server_key_iv: "41fb7b4a73d07dbd268278e0fd84aa71"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.789973 5112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:28.790720 5129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.790947 5112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:28.791033 5112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "2bc78a650b83422487b2b2f6f7125a57"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "d37d421b70d79f213310b63e7313acf2"
server_key_iv: "41fb7b4a73d07dbd268278e0fd84aa71"
server_key_version: "encryptionkey@0"
I20260808 10:03:28.791102 5112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:28.816900 5112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:28.819057 5112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:28.819214 5112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:28.819476 5112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:28.819785 5112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:28.819840 5112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.819885 5112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:28.819911 5112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:28.825675 5112 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:37799
I20260808 10:03:28.825714 5305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:37799 every 8 connection(s)
I20260808 10:03:28.826043 5112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:28.826701 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 5112
I20260808 10:03:28.826813 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:28.827075 25226 external_mini_cluster.cc:1428] Setting key f95768315afdb50b193a9c14593986d8
Aug 08 10:03:28 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:28.835925 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.827041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:37287 (local address 127.24.162.190:36427)
0808 10:03:28.827102 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:28.827104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:28.827116 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:28.827793 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:28.827795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:28.827808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:28.827878 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:28.829006 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.829607 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.830610 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.830827 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.830934 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.833036 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:28.833037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:28.833038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:28.833045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:28.834498 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.834938 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.834939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.834940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.835005 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:28.835269 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:28.835269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:28.835270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:28.835385 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:28.835462 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:28.835743 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 10:03:28.835837 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:03:28.836099 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:28.826762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:37287)
0808 10:03:28.826990 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:28.827041 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:28.827701 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:28.827948 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:28.827952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:28.828656 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:28.828914 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.828919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.829713 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.829714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.830514 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:28.830517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:28.830946 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:28.830947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:28.831082 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:28.831610 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:28.831619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:28.832939 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:28.834601 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.834608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.834610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.834834 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.835075 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:28.835076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:28.835077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:28.835192 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:28.835453 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:28.835460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:28.835598 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 10:03:28.835703 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:03:28.835863 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
I20260808 10:03:28.836557 5306 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36427
I20260808 10:03:28.836658 5306 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:28.836838 5306 heartbeater.cc:511] Master 127.24.162.190:36427 requested a full tablet report, sending...
I20260808 10:03:28.837224 4614 ts_manager.cc:194] Registered new tserver with Master: 2bc78a650b83422487b2b2f6f7125a57 (127.24.162.131:37799)
I20260808 10:03:28.837555 4614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:37287
I20260808 10:03:28.838979 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
Entry for principal oryx/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab.
WARNING: no policy specified for oryx/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
Entry for principal oryx/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab.
WARNING: no policy specified for oryx/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
Entry for principal oryx/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab.
I20260808 10:03:28.909807 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 4721
I20260808 10:03:28.919402 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 4924
I20260808 10:03:28.935014 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 5112
I20260808 10:03:28.943843 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 4559
I20260808 10:03:28.952929 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36427
--webserver_interface=127.24.162.190
--webserver_port=45123
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:29.034621 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:29.034816 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:29.034837 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:29.036013 5320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:29.036056 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:29.036070 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:29.036083 5320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:29.036134 5320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:29.037664 5320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:29.037930 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:29.038154 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:29.042986 5327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.043076 5328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.043111 5320 server_base.cc:1034] running on GCE node
W20260808 10:03:29.043011 5331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.043390 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:29.043673 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:29.044838 5320 hybrid_clock.cc:648] HybridClock initialized: now 1786183409044818 us; error 37 us; skew 500 ppm
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:29.047415 5320 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:29.047878 5320 webserver.cc:460] Webserver started at http://127.24.162.190:45123/ using document root <none> and password file <none>
I20260808 10:03:29.048121 5320 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:29.048177 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:29.049543 5320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:29.050191 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.050343 5320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:29.050434 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "b2083c8b089446b8a5004f355d4cea69"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "56243acf4a1d2d8b7a94c9daaf0c4c86"
server_key_iv: "ce05d460fa717aa959af45ae8a2bc7de"
server_key_version: "encryptionkey@0"
I20260808 10:03:29.050735 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:29.073980 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:29.076329 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:29.076444 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:29.080413 5320 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36427
I20260808 10:03:29.080492 5411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36427 every 8 connection(s)
I20260808 10:03:29.080925 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:29.082429 5412 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:03:29.083124 5412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Bootstrap starting.
I20260808 10:03:29.088229 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 5320
I20260808 10:03:29.088605 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:39975
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:36427
--webserver_port=38875
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:03:29.092553 5412 log.cc:824] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:29.094240 5412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:03:29.094544 5412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Bootstrap complete.
I20260808 10:03:29.096869 5412 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:29.097088 5412 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2083c8b089446b8a5004f355d4cea69, State: Initialized, Role: FOLLOWER
I20260808 10:03:29.097256 5412 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:29.097332 5412 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:29.097399 5412 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:29.097448 5412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:03:29.098438 5412 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:29.098527 5412 leader_election.cc:302] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2083c8b089446b8a5004f355d4cea69; no voters:
I20260808 10:03:29.098745 5412 leader_election.cc:288] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:03:29.098780 5416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:03:29.098935 5416 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [term 2 LEADER]: Becoming Leader. State: Replica: b2083c8b089446b8a5004f355d4cea69, State: Running, Role: LEADER
I20260808 10:03:29.099058 5412 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:29.099097 5416 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } }
I20260808 10:03:29.099658 5417 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2083c8b089446b8a5004f355d4cea69. Latest consensus state: current_term: 2 leader_uuid: "b2083c8b089446b8a5004f355d4cea69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } } }
I20260808 10:03:29.099758 5417 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:29.100104 5418 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b2083c8b089446b8a5004f355d4cea69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2083c8b089446b8a5004f355d4cea69" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36427 } } }
I20260808 10:03:29.100176 5418 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:29.101033 5442 catalog_manager.cc:1259] Loaded cluster ID: 3df208e23fc147f996f45b9b1d54ba30
I20260808 10:03:29.101083 5442 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: loading cluster ID for follower catalog manager: success
I20260808 10:03:29.102327 5442 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: acquiring CA information for follower catalog manager: success
I20260808 10:03:29.102654 5442 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:03:29.102733 5431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:29.102893 5431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:29.102979 5431 catalog_manager.cc:1259] Loaded cluster ID: 3df208e23fc147f996f45b9b1d54ba30
I20260808 10:03:29.103011 5431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:29.103170 5431 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:29.103242 5431 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2083c8b089446b8a5004f355d4cea69: Loaded TSK: 0
I20260808 10:03:29.103363 5431 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:03:29.173974 5414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:29.174187 5414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:29.174211 5414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:29.175411 5414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:29.175453 5414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:29.175479 5414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:29.176928 5414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.129.keytab
--principal=oryx/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=38875
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:29.177167 5414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:29.177415 5414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:29.191176 5455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.191291 5456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.191183 5458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.192251 5414 server_base.cc:1034] running on GCE node
I20260808 10:03:29.192546 5414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:29.192873 5414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:29.194087 5414 hybrid_clock.cc:648] HybridClock initialized: now 1786183409194064 us; error 37 us; skew 500 ppm
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183409, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:29.197289 5414 init.cc:377] Logged in from keytab as oryx/127.24.162.129@KRBTEST.COM (short username oryx)
I20260808 10:03:29.197822 5414 webserver.cc:460] Webserver started at http://127.24.162.129:38875/ using document root <none> and password file <none>
I20260808 10:03:29.198065 5414 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:29.198148 5414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:29.199810 5414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:29.200496 5466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.200677 5414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:29.200767 5414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "4649cb6dda12479fbc385f4003f650b2"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "87dfae21837da6fdfb2f7d5ba9e22772"
server_key_iv: "add1ea71c6d7510008f072571fe545fa"
server_key_version: "encryptionkey@0"
I20260808 10:03:29.201088 5414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:29.238448 5414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:29.240933 5414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:29.241050 5414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:29.241328 5414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:29.241665 5414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:29.241709 5414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.241732 5414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:29.241765 5414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.248215 5414 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:39975
I20260808 10:03:29.248255 5626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:39975 every 8 connection(s)
I20260808 10:03:29.248622 5414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:29.251314 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 5414
I20260808 10:03:29.251601 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:39651
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:36427
--webserver_port=35459
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.258083 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.249421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:42381)
0808 10:03:29.249642 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.249690 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.250207 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.250674 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.250678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.251353 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.251601 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.251605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.252766 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.252767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.254035 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.254039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.254753 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.254754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.254866 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.255464 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.255472 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.257852 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":108,"threads_started":1}
W20260808 10:03:29.258127 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.249492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:42381 (local address 127.24.162.190:36427)
0808 10:03:29.249688 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.249699 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.249787 (+ 88us) server_negotiation.cc:407] Connection header received
0808 10:03:29.250453 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.250457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.250499 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.250638 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.251777 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.252652 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.254160 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.254637 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.254751 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.257982 (+ 3231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:42381: BlockingRecv error: recv got EOF from 127.24.162.129:42381 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
W20260808 10:03:29.258320 5628 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:36427 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.162.190:36427: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.263716 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.258572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:45815)
0808 10:03:29.258708 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.258720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.258787 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.258955 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.258956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.259124 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.259187 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.259190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.259839 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.259840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.260773 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.260776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.261124 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.261126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.261201 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.261608 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.261614 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.263614 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260808 10:03:29.263747 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.258648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45815 (local address 127.24.162.190:36427)
0808 10:03:29.258707 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.258709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.258785 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:03:29.258851 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.258852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.258865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.258957 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.259290 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.259753 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.260870 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.261049 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.261100 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.263700 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:45815: BlockingRecv error: recv got EOF from 127.24.162.129:45815 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.268457 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.263988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:59631)
0808 10:03:29.264063 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.264081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.264110 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.264313 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.264315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.264469 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.264501 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.264503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.265087 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.265088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.265887 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.265889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.266198 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.266199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.266273 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.266633 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.266638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.268389 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 10:03:29.268608 5628 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:03:29.268750 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.264070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59631 (local address 127.24.162.190:36427)
0808 10:03:29.264168 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.264170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.264175 (+ 5us) server_negotiation.cc:407] Connection header received
0808 10:03:29.264212 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.264213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.264224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.264278 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.264579 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.265012 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.265965 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.266131 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.266179 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.268471 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:59631: BlockingRecv error: recv got EOF from 127.24.162.129:59631 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":121,"threads_started":1}
W20260808 10:03:29.339184 5638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:29.339388 5638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:29.339411 5638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:29.340567 5638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:29.340611 5638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:29.340667 5638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:29.342188 5638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.130.keytab
--principal=oryx/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:39651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=35459
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:29.342437 5638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:29.342669 5638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:29.347689 5650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.347839 5638 server_base.cc:1034] running on GCE node
W20260808 10:03:29.347708 5647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.347708 5648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.348137 5638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:29.348392 5638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:29.349553 5638 hybrid_clock.cc:648] HybridClock initialized: now 1786183409349544 us; error 48 us; skew 500 ppm
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183409, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:29.352025 5638 init.cc:377] Logged in from keytab as oryx/127.24.162.130@KRBTEST.COM (short username oryx)
I20260808 10:03:29.352463 5638 webserver.cc:460] Webserver started at http://127.24.162.130:35459/ using document root <none> and password file <none>
I20260808 10:03:29.352694 5638 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:29.352799 5638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:29.354190 5638 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:29.354811 5665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.355005 5638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:29.355118 5638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "c9c787ebb70047a8acb51b05d57695e8"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "96f9fb2c37bb7444a943c8f29074645b"
server_key_iv: "5a01a3e4fd887ef168317232bf94364e"
server_key_version: "encryptionkey@0"
I20260808 10:03:29.355461 5638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:29.385890 5638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:29.387945 5638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:29.388127 5638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:29.388382 5638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:29.388722 5638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:29.388782 5638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.388818 5638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:29.388870 5638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.394670 5638 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:39651
I20260808 10:03:29.394726 5828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:39651 every 8 connection(s)
I20260808 10:03:29.395082 5638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:29.398123 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 5638
I20260808 10:03:29.398444 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:37799
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:36427
--webserver_port=33419
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:03:29.403553 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.395794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:58451 (local address 127.24.162.190:36427)
0808 10:03:29.395840 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.395842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.396072 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:03:29.396530 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.396531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.396544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.396596 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.397497 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.398094 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.399264 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.399594 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.399671 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.403420 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:58451: BlockingRecv error: recv got EOF from 127.24.162.130:58451 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:03:29.403573 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.395788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:58451)
0808 10:03:29.395945 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.395982 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.396448 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.396646 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.396649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.397215 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.397417 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.397422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.398205 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.398206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.399181 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.399184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.399682 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.399683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.399791 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.400476 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.400487 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.403311 (+ 2824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
W20260808 10:03:29.403972 5829 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:36427 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.162.190:36427: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.409132 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.404215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:39623)
0808 10:03:29.404264 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.404274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.404312 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.404436 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.404437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.404600 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.404658 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.404660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.405339 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.405340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.406239 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.406242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.406564 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.406565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.406641 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.407105 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.407111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.409035 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:03:29.409154 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.404253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:39623 (local address 127.24.162.190:36427)
0808 10:03:29.404297 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.404298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.404307 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:29.404374 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.404375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.404382 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.404429 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.404733 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.405254 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.406324 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.406493 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.406541 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.409113 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:39623: BlockingRecv error: recv got EOF from 127.24.162.130:39623 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.413796 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.409388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:37221)
0808 10:03:29.409430 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.409440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.409476 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.409601 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.409601 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.409757 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.409795 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.409797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.410396 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.410397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.411289 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.411292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.411614 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.411615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.411681 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.412051 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.412058 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.413750 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 10:03:29.413914 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.409435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:37221 (local address 127.24.162.190:36427)
0808 10:03:29.409466 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.409468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.409482 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:03:29.409534 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.409535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.409541 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.409585 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.409879 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.410313 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.411370 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.411538 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.411589 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.413855 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:37221: BlockingRecv error: recv got EOF from 127.24.162.130:37221 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260808 10:03:29.414072 5829 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:03:29.486857 5833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:29.487099 5833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:29.487124 5833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:29.488250 5833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:29.488291 5833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:29.488317 5833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:29.490005 5833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/krb5kdc/oryx_127.24.162.131.keytab
--principal=oryx/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:37799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=33419
--tserver_master_addrs=127.24.162.190:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:29.490218 5833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:29.490402 5833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:29.495610 5846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.495591 5844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:29.495584 5841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:29.495913 5833 server_base.cc:1034] running on GCE node
I20260808 10:03:29.496196 5833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:29.496449 5833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:29.497622 5833 hybrid_clock.cc:648] HybridClock initialized: now 1786183409497614 us; error 58 us; skew 500 ppm
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183409, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:29.500476 5833 init.cc:377] Logged in from keytab as oryx/127.24.162.131@KRBTEST.COM (short username oryx)
I20260808 10:03:29.500968 5833 webserver.cc:460] Webserver started at http://127.24.162.131:33419/ using document root <none> and password file <none>
I20260808 10:03:29.501210 5833 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:29.501294 5833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:29.502653 5833 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:29.503443 5858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.503613 5833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.503748 5833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "2bc78a650b83422487b2b2f6f7125a57"
format_stamp: "Formatted at 2026-08-08 10:03:28 on dist-test-slave-nfb5"
server_key: "d37d421b70d79f213310b63e7313acf2"
server_key_iv: "41fb7b4a73d07dbd268278e0fd84aa71"
server_key_version: "encryptionkey@0"
I20260808 10:03:29.504106 5833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:29.528985 5833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:29.531620 5833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:29.531813 5833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:29.532073 5833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:29.532400 5833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:29.532457 5833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.532500 5833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:29.532545 5833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:29.538332 5833 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:37799
I20260808 10:03:29.538383 6024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:37799 every 8 connection(s)
I20260808 10:03:29.538779 5833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.545773 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 5833
W20260808 10:03:29.546253 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.539574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:43329 (local address 127.24.162.190:36427)
0808 10:03:29.539622 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.539627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.539847 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:03:29.540319 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.540320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.540330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.540392 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.541325 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.541831 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.542863 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.543099 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.543154 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.546178 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:43329: BlockingRecv error: recv got EOF from 127.24.162.131:43329 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:03:29.546316 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.539539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:43329)
0808 10:03:29.539708 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.539750 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.540234 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.540445 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.540449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.541030 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.541242 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.541247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.541936 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.541938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.542773 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.542776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.543175 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.543176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.543286 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.543881 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.543889 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.546070 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260808 10:03:29.546566 6025 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:36427 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.162.190:36427: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.551923 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.546787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:41927)
0808 10:03:29.546834 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.546844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.546878 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.547064 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.547065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.547227 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.547285 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.547286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.548136 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.548136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.548937 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.548939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.549352 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.549352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.549409 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.549820 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.549826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.551858 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:03:29.552045 5640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.546825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:41927 (local address 127.24.162.190:36427)
0808 10:03:29.546870 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.546871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.546878 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:03:29.546961 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.546962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.546971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.547039 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.547358 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.548057 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.549018 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.549262 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.549343 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.551973 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:41927: BlockingRecv error: recv got EOF from 127.24.162.131:41927 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 10:03:29.554780 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.546304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.24.162.190:36427)
0808 10:03:29.546345 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.546347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.546582 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:03:29.546633 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.546638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.546666 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.546722 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.547532 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.548173 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.549191 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.549445 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.549494 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.550999 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:29.551004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:29.551005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:29.551017 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:29.553341 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:29.553762 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:29.553764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:29.553765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:29.553857 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:29.554056 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:29.554056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:29.554057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:29.554246 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:29.554412 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:29.554547 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:03:29.554695 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:29 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:29.556885 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:29.552361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:52565)
0808 10:03:29.552409 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:29.552421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:29.552460 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:29.552595 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:29.552595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:29.552700 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:29.552725 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.552732 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.553548 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.553549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.554234 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:29.554236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.554697 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.554698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:29.554756 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.555152 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:29.555157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:29.556843 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:03:29.557008 6025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:03:29.557025 5640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:29.552437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:52565 (local address 127.24.162.190:36427)
0808 10:03:29.552479 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:29.552481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:29.552491 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:29.552511 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:29.552512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:29.552521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:29.552587 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:29.552783 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.553472 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.554312 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:29.554582 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:29.554645 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:29.556960 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:52565: BlockingRecv error: recv got EOF from 127.24.162.131:52565 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:30.273981 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:30.269101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:56207)
0808 10:03:30.269264 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:30.269275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:30.269328 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:30.269479 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:30.269480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:30.269648 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:30.269769 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.269772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.270466 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.270468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.271306 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.271309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.271651 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.271651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.271731 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.272168 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:30.272172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:30.273917 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
W20260808 10:03:30.274170 6032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:30.269196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:56207 (local address 127.24.162.190:36427)
0808 10:03:30.269341 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:30.269343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:30.269373 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:03:30.269394 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:30.269395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:30.269405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:30.269458 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:30.269841 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.270363 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.271382 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.271582 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.271639 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.274049 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:56207: BlockingRecv error: recv got EOF from 127.24.162.129:56207 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:30.419229 6033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:30.414528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:43335)
0808 10:03:30.414673 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:30.414685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:30.414739 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:30.414900 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:30.414901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:30.415106 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:30.415155 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.415157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.415825 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.415826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.416387 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.416389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.416713 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.416714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.416758 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.417274 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:30.417279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:30.419144 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
W20260808 10:03:30.419299 6032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:30.414622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:43335 (local address 127.24.162.190:36427)
0808 10:03:30.414686 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:30.414688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:30.414728 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:03:30.414801 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:30.414801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:30.414812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:30.414880 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:30.415252 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.415743 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.416461 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.416644 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.416694 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.419249 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:43335: BlockingRecv error: recv got EOF from 127.24.162.130:43335 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:30 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:30.562496 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:30.557433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:36813)
0808 10:03:30.557578 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:30.557588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:30.557636 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:30.557810 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:30.557811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:30.557980 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:30.558031 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.558033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.558676 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.558677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.559317 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:30.559320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.559645 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.559646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:30.559717 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.560170 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:30.560174 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:30.562385 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
W20260808 10:03:30.562529 6032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:30.557559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:36813 (local address 127.24.162.190:36427)
0808 10:03:30.557621 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:30.557623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:30.557632 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:30.557709 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:30.557710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:30.557726 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:30.557792 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:30.558093 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.558609 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.559390 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:30.559578 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:30.559635 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:30.562462 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:36813: BlockingRecv error: recv got EOF from 127.24.162.131:36813 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:31.279577 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:31.274537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:38475)
0808 10:03:31.274670 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:31.274681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:31.274730 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:31.274905 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:31.274905 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:31.275154 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:31.275206 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.275209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.275974 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.275976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.276504 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.276505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.276929 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.276930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.276972 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.277529 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:31.277534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:31.279515 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
W20260808 10:03:31.279778 6038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:31.274605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:38475 (local address 127.24.162.190:36427)
0808 10:03:31.274776 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:31.274778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:31.274787 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:31.274808 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:31.274809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:31.274820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:31.274890 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:31.275284 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.275889 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.276599 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.276854 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.276919 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.279673 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:38475: BlockingRecv error: recv got EOF from 127.24.162.129:38475 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:31.424489 6039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:31.419785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:51199)
0808 10:03:31.419956 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:31.419967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:31.420030 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:31.420155 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:31.420155 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:31.420324 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:31.420375 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.420377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.421076 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.421077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.421660 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.421663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.422033 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.422034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.422101 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.422617 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:31.422622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:31.424432 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260808 10:03:31.424639 6038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:31.419841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:51199 (local address 127.24.162.190:36427)
0808 10:03:31.419917 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:31.419919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:31.420030 (+ 111us) server_negotiation.cc:407] Connection header received
0808 10:03:31.420086 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:31.420087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:31.420100 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:31.420141 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:31.420435 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.420968 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.421746 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.421954 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.422033 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.424548 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:51199: BlockingRecv error: recv got EOF from 127.24.162.130:51199 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:31 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:31.567652 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:31.563063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:37121)
0808 10:03:31.563203 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:31.563213 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:31.563265 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:31.563417 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:31.563418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:31.563589 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:31.563638 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.563640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.564279 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.564280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.564825 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:31.564827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.565175 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.565176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:31.565224 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.565702 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:31.565708 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:31.567581 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
W20260808 10:03:31.567776 6038 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:31.563185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:37121 (local address 127.24.162.190:36427)
0808 10:03:31.563235 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:31.563238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:31.563246 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:31.563329 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:31.563330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:31.563342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:31.563396 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:31.563701 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.564198 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.564898 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:31.565094 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:31.565157 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:31.567680 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:37121: BlockingRecv error: recv got EOF from 127.24.162.131:37121 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:32.284865 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:32.280136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:36499)
0808 10:03:32.280312 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:32.280324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:32.280386 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:32.280534 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:32.280535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:32.280720 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:32.280771 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.280773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.281420 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.281421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.281989 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.281991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.282391 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.282392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.282432 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.283014 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:32.283019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:32.284778 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
W20260808 10:03:32.284941 6043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:32.280270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:36499 (local address 127.24.162.190:36427)
0808 10:03:32.280410 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:32.280412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:32.280420 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:32.280442 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:32.280443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:32.280454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:32.280515 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:32.280852 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.281353 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.282090 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.282317 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.282380 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.284884 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:36499: BlockingRecv error: recv got EOF from 127.24.162.129:36499 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:32.429684 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:32.424953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:45879)
0808 10:03:32.425096 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:32.425107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:32.425171 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:32.425322 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:32.425323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:32.425496 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:32.425557 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.425558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.426192 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.426192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.426756 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.426758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.427091 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.427091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.427164 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.427779 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:32.427785 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:32.429577 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260808 10:03:32.429739 6043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:32.425057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:45879 (local address 127.24.162.190:36427)
0808 10:03:32.425121 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:32.425123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:32.425167 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:03:32.425234 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:32.425234 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:32.425245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:32.425288 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:32.425624 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.426115 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.426832 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.427031 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.427083 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.429672 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:45879: BlockingRecv error: recv got EOF from 127.24.162.130:45879 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:32 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:32.573395 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:32.568181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:52181)
0808 10:03:32.568414 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:32.568428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:32.568543 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:32.568705 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:32.568707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:32.568978 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:32.569027 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.569030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.569747 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.569748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.570285 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:32.570287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.570674 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.570675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:32.570735 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.571271 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:32.571277 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:32.573334 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":127,"threads_started":1}
W20260808 10:03:32.573563 6043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:32.568309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:52181 (local address 127.24.162.190:36427)
0808 10:03:32.568416 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:32.568418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:32.568521 (+ 103us) server_negotiation.cc:407] Connection header received
0808 10:03:32.568604 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:32.568605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:32.568621 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:32.568699 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:32.569119 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.569660 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.570391 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:32.570585 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:32.570674 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:32.573474 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:52181: BlockingRecv error: recv got EOF from 127.24.162.131:52181 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:33.290124 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:33.285401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:40057)
0808 10:03:33.285569 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:33.285581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:33.285636 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:33.285775 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:33.285775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:33.285949 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:33.286000 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.286002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.286633 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.286633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.287198 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.287199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.287531 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.287532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.287572 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.288119 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:33.288125 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:33.290025 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
W20260808 10:03:33.290206 6050 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:33.285496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:40057 (local address 127.24.162.190:36427)
0808 10:03:33.285617 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:33.285619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:33.285628 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:33.285695 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:33.285696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:33.285708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:33.285757 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:33.286066 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.286570 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.287269 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.287465 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.287520 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.290108 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:40057: BlockingRecv error: recv got EOF from 127.24.162.129:40057 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:33.438004 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:33.430221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:44961)
0808 10:03:33.430423 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:33.430437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:33.430526 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:33.430692 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:33.430693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:33.430920 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:33.431018 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.431020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.431803 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.431805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.432354 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.432356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.432745 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.432746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.432806 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.436150 (+ 3344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:33.436155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:33.437945 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
W20260808 10:03:33.438146 6050 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:33.430311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:44961 (local address 127.24.162.190:36427)
0808 10:03:33.430406 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:33.430409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:33.430506 (+ 97us) server_negotiation.cc:407] Connection header received
0808 10:03:33.430618 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:33.430619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:33.430632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:33.430687 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:33.431100 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.431724 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.432446 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.432660 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.432744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.438057 (+ 5313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:44961: BlockingRecv error: recv got EOF from 127.24.162.130:44961 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:33 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:33.578298 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:33.573896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:37561)
0808 10:03:33.574072 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:33.574084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:33.574149 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:33.574289 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:33.574290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:33.574459 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:33.574510 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.574512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.575153 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.575153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.575676 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:33.575677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.575992 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.575993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:33.576032 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.576476 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:33.576480 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:33.578239 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
W20260808 10:03:33.578430 6050 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:33.574011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:37561 (local address 127.24.162.190:36427)
0808 10:03:33.574066 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:33.574068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:33.574135 (+ 67us) server_negotiation.cc:407] Connection header received
0808 10:03:33.574193 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:33.574193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:33.574206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:33.574266 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:33.574580 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.575089 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.575755 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:33.575931 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:33.575980 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:33.578345 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:37561: BlockingRecv error: recv got EOF from 127.24.162.131:37561 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:34.295459 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:34.290624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:46083)
0808 10:03:34.290776 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:34.290787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:34.290837 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:34.291023 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:34.291023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:34.291206 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:34.291257 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.291259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.291923 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.291924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.292458 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.292459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.292817 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.292818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.292862 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.293436 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:34.293441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:34.295397 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
W20260808 10:03:34.295615 6058 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:34.290699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:46083 (local address 127.24.162.190:36427)
0808 10:03:34.290877 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:34.290878 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:34.290886 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:34.290908 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:34.290909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:34.290937 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:34.291000 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:34.291338 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.291843 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.292538 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.292743 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.292798 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.295507 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:46083: BlockingRecv error: recv got EOF from 127.24.162.129:46083 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:34.443102 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:34.438495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:57249)
0808 10:03:34.438651 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:34.438662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:34.438720 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:34.438856 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:34.438856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:34.439040 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:34.439108 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.439110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.439808 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.439809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.440334 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.440335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.440693 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.440694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.440732 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.441210 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:34.441215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:34.443043 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260808 10:03:34.443298 6058 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:34.438618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:57249 (local address 127.24.162.190:36427)
0808 10:03:34.438662 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:34.438664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:34.438706 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:03:34.438773 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:34.438774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:34.438786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:34.438827 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:34.439193 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.439722 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.440419 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.440613 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.440680 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.443201 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:57249: BlockingRecv error: recv got EOF from 127.24.162.130:57249 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:34 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:34.583932 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:34.578884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:45945)
0808 10:03:34.579084 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:34.579096 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:34.579180 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:34.579384 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:34.579386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:34.579615 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:34.579679 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.579681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.580350 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.580351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.580907 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:34.580908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.581339 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.581340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:34.581391 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.581891 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:34.581896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:34.583871 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":120,"threads_started":1}
W20260808 10:03:34.584052 6058 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:34.579003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:45945 (local address 127.24.162.190:36427)
0808 10:03:34.579062 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:34.579065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:34.579166 (+ 101us) server_negotiation.cc:407] Connection header received
0808 10:03:34.579267 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:34.579269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:34.579285 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:34.579347 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:34.579759 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.580256 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.580992 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:34.581239 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:34.581313 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:34.583973 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:45945: BlockingRecv error: recv got EOF from 127.24.162.131:45945 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:35.301270 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:35.296079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:57229)
0808 10:03:35.296226 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:35.296238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:35.296312 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:35.296597 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:35.296598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:35.296801 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:35.296867 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.296869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.297535 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.297536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.298078 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.298079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.298443 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.298444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.298514 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.299133 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:35.299138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:35.301174 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
W20260808 10:03:35.301352 6062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:35.296108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:57229 (local address 127.24.162.190:36427)
0808 10:03:35.296250 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:35.296252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:35.296308 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:03:35.296410 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:35.296410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:35.296438 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:35.296515 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:35.296940 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.297464 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.298149 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.298352 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.298406 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.301279 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:57229: BlockingRecv error: recv got EOF from 127.24.162.129:57229 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:35.448838 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:35.443622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:35359)
0808 10:03:35.444010 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:35.444021 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:35.444092 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:35.444227 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:35.444228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:35.444406 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:35.444468 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.444469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.445110 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.445111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.445627 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.445629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.445992 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.445993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.446057 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.446515 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:35.446520 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:35.448778 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":310,"threads_started":1}
W20260808 10:03:35.448984 6062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:35.443791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:35359 (local address 127.24.162.190:36427)
0808 10:03:35.443873 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:35.443875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:35.444101 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:03:35.444133 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:35.444134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:35.444148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:35.444204 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:35.444537 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.445026 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.445743 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.445926 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.445976 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.448902 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:35359: BlockingRecv error: recv got EOF from 127.24.162.130:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:35 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:35.589640 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:35.584520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:47389)
0808 10:03:35.584806 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:35.584831 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:35.584919 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:35.585090 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:35.585091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:35.585283 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:35.585332 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.585333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.586092 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.586093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.586644 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:35.586646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.587094 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.587095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:35.587155 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.587674 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:35.587680 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:35.589581 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":77,"threads_started":1}
W20260808 10:03:35.589823 6062 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:35.584586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:47389 (local address 127.24.162.190:36427)
0808 10:03:35.584762 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:35.584765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:35.584900 (+ 135us) server_negotiation.cc:407] Connection header received
0808 10:03:35.584978 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:35.584979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:35.584996 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:35.585068 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:35.585417 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.586003 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.586740 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:35.587008 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:35.587092 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:35.589733 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:47389: BlockingRecv error: recv got EOF from 127.24.162.131:47389 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:36.306324 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:36.301718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:35993)
0808 10:03:36.301863 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:36.301883 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:36.301932 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:36.302060 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:36.302061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:36.302226 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:36.302278 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.302280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.302996 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.302996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.303569 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.303572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.303952 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.303953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.304004 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.304442 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:36.304446 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:36.306263 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
W20260808 10:03:36.306540 6067 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:36.301798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:35993 (local address 127.24.162.190:36427)
0808 10:03:36.301929 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:36.301931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:36.301944 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:03:36.301970 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:36.301970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:36.301979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:36.302041 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:36.302345 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.302873 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.303636 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.303848 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.303940 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.306361 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:35993: BlockingRecv error: recv got EOF from 127.24.162.129:35993 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:36.454032 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:36.449301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:43431)
0808 10:03:36.449486 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:36.449498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:36.449563 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:36.449733 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:36.449734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:36.449897 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:36.449949 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.449950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.450590 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.450590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.451227 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.451229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.451557 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.451558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.451597 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.452081 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:36.452085 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:36.453973 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":129,"threads_started":1}
W20260808 10:03:36.454139 6067 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:36.449358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:43431 (local address 127.24.162.190:36427)
0808 10:03:36.449445 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:36.449447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:36.449566 (+ 119us) server_negotiation.cc:407] Connection header received
0808 10:03:36.449611 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:36.449612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:36.449623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:36.449676 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:36.450014 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.450517 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.451305 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.451494 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.451545 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.454069 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:43431: BlockingRecv error: recv got EOF from 127.24.162.130:43431 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:36 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:36.595026 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:36.590156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:34491)
0808 10:03:36.590326 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:36.590338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:36.590396 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:36.590532 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:36.590533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:36.590703 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:36.590756 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.590758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.591497 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.591498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.592013 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:36.592014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.592398 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.592399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:36.592446 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.592908 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:36.592912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:36.594961 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260808 10:03:36.595137 6067 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:36.590255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:34491 (local address 127.24.162.190:36427)
0808 10:03:36.590331 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:36.590333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:36.590381 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:03:36.590440 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:36.590441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:36.590453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:36.590512 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:36.590835 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.591420 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.592103 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:36.592301 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:36.592380 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:36.595068 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:34491: BlockingRecv error: recv got EOF from 127.24.162.131:34491 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:37.311357 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:37.306790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:35095)
0808 10:03:37.306982 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:37.306993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:37.307045 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:37.307187 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:37.307188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:37.307350 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:37.307401 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.307403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.308035 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.308036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.308550 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.308551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.308896 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.308897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.308954 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.309505 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:37.309510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:37.311298 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
W20260808 10:03:37.311467 6071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:37.306944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:35095 (local address 127.24.162.190:36427)
0808 10:03:37.307069 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:37.307071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:37.307079 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:37.307100 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:37.307100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:37.307110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:37.307171 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:37.307464 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.307966 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.308620 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.308825 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.308880 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.311391 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:35095: BlockingRecv error: recv got EOF from 127.24.162.129:35095 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:37.459685 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:37.454596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:49917)
0808 10:03:37.454765 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:37.454777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:37.454841 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:37.455031 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:37.455032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:37.455196 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:37.455246 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.455248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.456053 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.456054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.456558 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.456559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.456927 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.456927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.456966 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.457432 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:37.457437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:37.459587 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
W20260808 10:03:37.459725 6071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:37.454653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:49917 (local address 127.24.162.190:36427)
0808 10:03:37.454796 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:37.454798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:37.454812 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:03:37.454905 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:37.454905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:37.454952 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:37.455015 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:37.455320 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.455985 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.456640 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.456854 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.456921 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.459665 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:49917: BlockingRecv error: recv got EOF from 127.24.162.130:49917 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:37 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:37.600589 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:37.595510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:58113)
0808 10:03:37.595736 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:37.595747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:37.595829 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:37.595962 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:37.595963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:37.596150 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:37.596202 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.596204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.596885 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.596886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.597407 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:37.597409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.597775 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.597776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:37.597844 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.598357 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:37.598363 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:37.600492 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260808 10:03:37.600651 6071 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:37.595606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:58113 (local address 127.24.162.190:36427)
0808 10:03:37.595680 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:37.595683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:37.595813 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:03:37.595886 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:37.595887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:37.595904 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:37.595957 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:37.596298 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.596797 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.597498 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:37.597700 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:37.597777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:37.600570 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:58113: BlockingRecv error: recv got EOF from 127.24.162.131:58113 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:38.316385 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:38.311884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:50285)
0808 10:03:38.312036 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:38.312047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:38.312103 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:38.312254 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:38.312255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:38.312422 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:38.312474 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.312476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.313095 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.313096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.313616 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.313617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.313934 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.313935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.313974 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.314436 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:38.314441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:38.316326 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
W20260808 10:03:38.316519 6076 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:38.311932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:50285 (local address 127.24.162.190:36427)
0808 10:03:38.312061 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:38.312063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:38.312087 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:03:38.312157 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:38.312157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:38.312169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:38.312231 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:38.312536 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.313025 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.313689 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.313871 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.313921 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.316428 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:50285: BlockingRecv error: recv got EOF from 127.24.162.129:50285 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:38.468891 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:38.460101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:41643)
0808 10:03:38.460270 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:38.460282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:38.460392 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:38.460546 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:38.460548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:38.460760 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:38.460809 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.460811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.461477 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.461478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.462001 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.462003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.462376 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.462377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.462437 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.466868 (+ 4431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:38.466874 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:38.468795 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
W20260808 10:03:38.468954 6076 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:38.460182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:41643 (local address 127.24.162.190:36427)
0808 10:03:38.460257 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:38.460260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:38.460368 (+ 108us) server_negotiation.cc:407] Connection header received
0808 10:03:38.460453 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:38.460454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:38.460474 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:38.460540 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:38.460897 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.461395 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.462098 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.462302 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.462378 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.468891 (+ 6513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:41643: BlockingRecv error: recv got EOF from 127.24.162.130:41643 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:38 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:38.606132 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:38.601296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:55255)
0808 10:03:38.601425 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:38.601437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:38.601520 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:38.601669 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:38.601671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:38.601892 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:38.601952 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.601954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.602654 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.602654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.603331 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:38.603333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.603640 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.603641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:38.603680 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.604181 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:38.604186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:38.606074 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
W20260808 10:03:38.606258 6076 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:38.601372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:55255 (local address 127.24.162.190:36427)
0808 10:03:38.601440 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:38.601442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:38.601515 (+ 73us) server_negotiation.cc:407] Connection header received
0808 10:03:38.601569 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:38.601570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:38.601585 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:38.601651 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:38.602035 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.602583 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.603399 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:38.603576 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:38.603631 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:38.606166 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:55255: BlockingRecv error: recv got EOF from 127.24.162.131:55255 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:39.321663 6079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:39.316907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:40917)
0808 10:03:39.317059 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:39.317071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:39.317131 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:39.317267 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:39.317268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:39.317444 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:39.317498 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.317499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.318134 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.318134 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.318651 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.318652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.319004 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.319004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.319044 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.319584 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:39.319590 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:39.321562 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
W20260808 10:03:39.321781 6080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:39.316994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:40917 (local address 127.24.162.190:36427)
0808 10:03:39.317128 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:39.317130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:39.317139 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:39.317187 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:39.317187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:39.317199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:39.317250 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:39.317561 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.318065 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.318728 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.318940 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.318994 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.321652 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:40917: BlockingRecv error: recv got EOF from 127.24.162.129:40917 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:39.474548 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:39.469428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:36461)
0808 10:03:39.469625 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:39.469638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:39.469710 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:39.469887 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:39.469889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:39.470148 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:39.470210 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.470212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.470882 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.470883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.471463 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.471465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.472101 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.472102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.472167 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.472659 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:39.472664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:39.474451 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":119,"threads_started":1}
W20260808 10:03:39.474618 6080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:39.469489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:36461 (local address 127.24.162.190:36427)
0808 10:03:39.469567 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:39.469569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:39.469768 (+ 199us) server_negotiation.cc:407] Connection header received
0808 10:03:39.469801 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:39.469802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:39.469817 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:39.469875 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:39.470291 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.470797 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.471552 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.471972 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.472098 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.474546 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:36461: BlockingRecv error: recv got EOF from 127.24.162.130:36461 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:39 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:39.611495 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:39.606606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:40943)
0808 10:03:39.606781 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:39.606793 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:39.606878 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:39.607051 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:39.607051 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:39.607222 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:39.607273 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.607275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.607991 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.607992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.608477 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:39.608479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.608805 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.608806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:39.608855 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.609332 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:39.609336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:39.611378 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
W20260808 10:03:39.611526 6080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:39.606734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:40943 (local address 127.24.162.190:36427)
0808 10:03:39.606801 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:39.606803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:39.606861 (+ 58us) server_negotiation.cc:407] Connection header received
0808 10:03:39.606965 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:39.606965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:39.606980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:39.607032 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:39.607343 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.607926 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.608542 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:39.608744 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:39.608797 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:39.611455 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:40943: BlockingRecv error: recv got EOF from 127.24.162.131:40943 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:40.327418 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:40.322137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:40223)
0808 10:03:40.322307 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:40.322327 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:40.322381 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:40.322513 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:40.322513 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:40.322687 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:40.322737 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.322738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.323402 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.323403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.324106 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.324109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.324672 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.324673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.324734 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.325256 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:40.325262 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:40.327302 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
W20260808 10:03:40.327459 6085 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:40.322245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:40223 (local address 127.24.162.190:36427)
0808 10:03:40.322358 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:40.322360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:40.322368 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:03:40.322437 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:40.322437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:40.322448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:40.322497 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:40.322804 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.323327 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.324220 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.324577 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.324652 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.327385 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:40223: BlockingRecv error: recv got EOF from 127.24.162.129:40223 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:03:40.480165 6085 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:40.475213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:52035 (local address 127.24.162.190:36427)
0808 10:03:40.475266 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:40.475269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:40.475320 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:03:40.475386 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:40.475387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:40.475399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:40.475448 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:40.475769 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.476281 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.477260 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.477453 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.477511 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.480103 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:52035: BlockingRecv error: recv got EOF from 127.24.162.130:52035 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:03:40.480137 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:40.475110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:52035)
0808 10:03:40.475253 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:40.475266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:40.475337 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:40.475475 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:40.475476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:40.475653 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:40.475703 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.475704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.476354 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.476355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.477182 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.477185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.477517 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.477517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.477555 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.478023 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:40.478028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:40.480017 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:40 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:40.617009 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:40.612065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:41881)
0808 10:03:40.612230 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:40.612250 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:40.612307 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:40.612469 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:40.612470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:40.612637 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:40.612697 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.612699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.613339 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.613340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.613878 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:40.613880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.614236 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.614237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:40.614290 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.614835 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:40.614841 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:40.616903 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
W20260808 10:03:40.617235 6085 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:40.612145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:41881 (local address 127.24.162.190:36427)
0808 10:03:40.612213 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:40.612215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:40.612306 (+ 91us) server_negotiation.cc:407] Connection header received
0808 10:03:40.612367 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:40.612367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:40.612380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:40.612453 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:40.612768 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.613265 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.613954 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:40.614134 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:40.614184 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:40.617010 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:41881: BlockingRecv error: recv got EOF from 127.24.162.131:41881 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:41.333062 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:41.328127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:55817)
0808 10:03:41.328286 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:41.328298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:41.328360 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:41.328503 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:41.328503 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:41.328725 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:41.328775 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.328777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.329486 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.329487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.330079 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.330081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.330458 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.330459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.330515 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.331076 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:41.331081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:41.332998 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
W20260808 10:03:41.333205 6090 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:41.328215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:55817 (local address 127.24.162.190:36427)
0808 10:03:41.328340 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:41.328342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:41.328353 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:41.328409 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:41.328410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:41.328421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:41.328480 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:41.328885 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.329406 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.330171 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.330385 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.330443 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.333095 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:55817: BlockingRecv error: recv got EOF from 127.24.162.129:55817 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:41.485854 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:41.480726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:56823)
0808 10:03:41.480898 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:41.480913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:41.480981 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:41.481166 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:41.481168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:41.481369 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:41.481417 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.481419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.482099 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.482100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.482895 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.482897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.483309 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.483310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.483367 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.483862 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:41.483866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:41.485792 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260808 10:03:41.485999 6090 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:41.480862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:56823 (local address 127.24.162.190:36427)
0808 10:03:41.480935 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:41.480937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:41.480962 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:03:41.481076 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:41.481077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:41.481095 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:41.481158 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:41.481507 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.482009 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.483003 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.483203 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.483309 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.485908 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:56823: BlockingRecv error: recv got EOF from 127.24.162.130:56823 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:41 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:41.622174 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:41.617564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:34101)
0808 10:03:41.617724 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:41.617734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:41.617785 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:41.617939 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:41.617939 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:41.618118 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:41.618166 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.618168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.618838 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.618839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.619410 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:41.619412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.619785 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.619786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:41.619830 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.620348 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:41.620353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:41.622114 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
W20260808 10:03:41.622337 6090 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:41.617646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:34101 (local address 127.24.162.190:36427)
0808 10:03:41.617764 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:41.617766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:41.617776 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:41.617845 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:41.617846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:41.617859 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:41.617918 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:41.618234 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.618767 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.619519 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:41.619713 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:41.619779 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:41.622255 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:34101: BlockingRecv error: recv got EOF from 127.24.162.131:34101 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:42.338729 6097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:42.333566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:52901)
0808 10:03:42.333751 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:42.333768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:42.333872 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:42.334046 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:42.334047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:42.334224 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:42.334275 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.334277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.335020 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.335021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.335556 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.335558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.336141 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.336143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.336208 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.336744 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:42.336748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:42.338669 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
W20260808 10:03:42.338939 6098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:42.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:52901 (local address 127.24.162.190:36427)
0808 10:03:42.333905 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:42.333907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:42.333917 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:03:42.333941 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:42.333942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:42.333954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:42.334025 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:42.334351 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.334941 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.335651 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.336040 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.336120 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.338815 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:52901: BlockingRecv error: recv got EOF from 127.24.162.129:52901 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:42.491966 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:42.486550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:58365)
0808 10:03:42.486798 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:42.486811 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:42.486918 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:42.487179 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:42.487181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:42.487438 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:42.487499 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.487501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.488286 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.488287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.488819 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.488821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.489181 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.489182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.489223 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.489769 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:42.489775 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:42.491903 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
W20260808 10:03:42.492161 6098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:42.486741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:58365 (local address 127.24.162.190:36427)
0808 10:03:42.486830 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:42.486832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:42.486880 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:03:42.487055 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:42.487056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:42.487071 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:42.487142 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:42.487594 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.488193 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.488914 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.489109 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.489167 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.492071 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:58365: BlockingRecv error: recv got EOF from 127.24.162.130:58365 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:42 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:42.629294 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:42.622782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:53801)
0808 10:03:42.622929 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:42.622949 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:42.623012 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:42.623235 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:42.623237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:42.623483 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:42.623562 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.623564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.624576 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.624577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.625495 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:42.625500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.626090 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.626091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:42.626166 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.626719 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:42.626725 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:42.629181 (+ 2456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
W20260808 10:03:42.629338 6098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:42.622863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:53801 (local address 127.24.162.190:36427)
0808 10:03:42.622983 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:42.622985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:42.622997 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:42.623074 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:42.623075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:42.623113 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:42.623185 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:42.623650 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.624483 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.625602 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:42.625976 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:42.626069 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:42.629262 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:53801: BlockingRecv error: recv got EOF from 127.24.162.131:53801 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:43.343917 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:43.339171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:52659)
0808 10:03:43.339301 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:43.339313 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:43.339387 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:43.339521 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:43.339522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:43.339691 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:43.339741 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.339743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.340386 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.340387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.340919 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.340920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.341297 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.341298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.341341 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.341991 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:43.341997 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:43.343858 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
W20260808 10:03:43.344071 6103 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:43.339246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:52659 (local address 127.24.162.190:36427)
0808 10:03:43.339392 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:43.339394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:43.339405 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:43.339429 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:43.339430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:43.339448 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:43.339507 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:43.339807 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.340315 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.341019 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.341223 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.341295 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.343969 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:52659: BlockingRecv error: recv got EOF from 127.24.162.129:52659 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:43.501487 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:43.492514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:36211)
0808 10:03:43.492765 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:43.492802 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:43.492893 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:43.493087 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:43.493089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:43.493284 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:43.493366 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.493368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.494093 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.494094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.494643 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.494645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.495017 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.495018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.495072 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.499422 (+ 4350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:43.499427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:43.501415 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260808 10:03:43.501631 6103 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:43.492630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:36211 (local address 127.24.162.190:36427)
0808 10:03:43.492742 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:43.492744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:43.492877 (+ 133us) server_negotiation.cc:407] Connection header received
0808 10:03:43.492983 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:43.492984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:43.492998 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:43.493053 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:43.493454 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.494021 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.494736 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.494950 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.495003 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.501544 (+ 6541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:36211: BlockingRecv error: recv got EOF from 127.24.162.130:36211 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:43 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.131@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:43.635679 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:43.630103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.131:55179)
0808 10:03:43.630294 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:43.630306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:43.630404 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:43.630580 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:43.630582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:43.630810 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:43.630861 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.630863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.631643 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.631645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.632289 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:43.632292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.632695 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.632697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:43.632766 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.633344 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:43.633349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:43.635579 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
W20260808 10:03:43.635725 6103 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:43.630166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:55179 (local address 127.24.162.190:36427)
0808 10:03:43.630248 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:43.630251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:43.630390 (+ 139us) server_negotiation.cc:407] Connection header received
0808 10:03:43.630470 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:43.630471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:43.630490 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:43.630574 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:43.630990 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.631544 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.632390 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:43.632603 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:43.632695 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:43.635660 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.131:55179: BlockingRecv error: recv got EOF from 127.24.162.131:55179 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 10:03:44 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:44 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.129@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:44.349349 6106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:44.344631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.129:35283)
0808 10:03:44.344805 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:44.344816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:44.344870 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:44.345005 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:44.345006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:44.345175 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:44.345224 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.345226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.345892 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.345892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.346407 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.346408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.346783 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.346784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.346824 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.347379 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:44.347384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:44.349276 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
W20260808 10:03:44.349424 6108 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:44.344738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:35283 (local address 127.24.162.190:36427)
0808 10:03:44.344845 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:44.344847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:44.344859 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:03:44.344918 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:44.344918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:44.344928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:44.344986 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:44.345291 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.345823 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.346502 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.346710 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.346772 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.349372 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.129:35283: BlockingRecv error: recv got EOF from 127.24.162.129:35283 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Aug 08 10:03:44 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:03:44 dist-test-slave-nfb5 krb5kdc[4541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.162.130@KRBTEST.COM for oryx/127.24.162.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:03:44.507026 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:44.502009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36427 (local address 127.24.162.130:32827)
0808 10:03:44.502205 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:44.502217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:44.502304 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:44.502465 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:44.502466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:44.502656 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:44.502705 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.502706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.503533 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.503534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.504119 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.504121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.504551 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.504551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.504611 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.505119 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:44.505123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:44.506905 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.162.190:36427: Server oryx/127.24.162.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
W20260808 10:03:44.507083 6108 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:44.502110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:32827 (local address 127.24.162.190:36427)
0808 10:03:44.502186 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:44.502189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:44.502286 (+ 97us) server_negotiation.cc:407] Connection header received
0808 10:03:44.502366 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:44.502367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:44.502381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:44.502459 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:44.502795 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.503446 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.504239 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.504460 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.504549 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.507017 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.162.130:32827: BlockingRecv error: recv got EOF from 127.24.162.130:32827 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:03:44.552197 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 5414
I20260808 10:03:44.567569 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 5638
I20260808 10:03:44.574419 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 5833
I20260808 10:03:44.582530 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 5320
2026-08-08T10:03:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17733 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T10:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:44Z Disabled control of system clock
I20260808 10:03:44.614506 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38363
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:03:44.696228 6113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:44.696473 6113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:44.696509 6113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:44.697671 6113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:44.697750 6113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:44.697772 6113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:44.697793 6113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:44.697822 6113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:44.699211 6113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:38363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:44.699494 6113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:44.699798 6113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:44.704607 6119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:44.704650 6120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:44.704629 6124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:44.704895 6113 server_base.cc:1034] running on GCE node
I20260808 10:03:44.705152 6113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:44.705430 6113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:44.706597 6113 hybrid_clock.cc:648] HybridClock initialized: now 1786183424706564 us; error 48 us; skew 500 ppm
I20260808 10:03:44.707873 6113 webserver.cc:460] Webserver started at http://127.24.162.190:41543/ using document root <none> and password file <none>
I20260808 10:03:44.708127 6113 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:44.708199 6113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:44.708349 6113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:44.709386 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "fdf783afaea649e69dbca7a15145609a"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "26a80176c3c0729cab2424e854b1521a"
server_key_iv: "c3c44d397a6695c795038f15b7e01001"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.709746 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "fdf783afaea649e69dbca7a15145609a"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "26a80176c3c0729cab2424e854b1521a"
server_key_iv: "c3c44d397a6695c795038f15b7e01001"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.711076 6113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:44.711843 6131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:44.712045 6113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:44.712150 6113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "fdf783afaea649e69dbca7a15145609a"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "26a80176c3c0729cab2424e854b1521a"
server_key_iv: "c3c44d397a6695c795038f15b7e01001"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.712239 6113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:44.732595 6113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:44.732956 6113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:44.733127 6113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:44.737485 6113 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:38363
I20260808 10:03:44.737526 6210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:38363 every 8 connection(s)
I20260808 10:03:44.737914 6113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:44.738669 6212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:44.738855 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6113
I20260808 10:03:44.738960 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:44.739197 25226 external_mini_cluster.cc:1428] Setting key 0c822b5ce9ea58b6810e0ec27e9b7830
I20260808 10:03:44.742627 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a: Bootstrap starting.
I20260808 10:03:44.743304 6212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:44.743690 6212 log.cc:824] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:44.744478 6212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a: No bootstrap required, opened a new log
I20260808 10:03:44.744791 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:44.739803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.24.162.190:38363)
0808 10:03:44.740024 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:44.740031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:44.740056 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:03:44.740434 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:44.740440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:44.740498 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:44.740609 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:44.740872 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.741713 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.742795 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.743274 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.743402 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.744135 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:44.744139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:44.744139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:44.744317 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:44.744350 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:44.744361 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:44.744433 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:44.744508 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:03:44.744590 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260808 10:03:44.746428 6212 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } }
I20260808 10:03:44.746585 6212 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:44.746627 6212 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf783afaea649e69dbca7a15145609a, State: Initialized, Role: FOLLOWER
I20260808 10:03:44.746780 6212 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } }
I20260808 10:03:44.746937 6212 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:44.746995 6212 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:44.747051 6212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:44.747609 6212 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } }
I20260808 10:03:44.747735 6212 leader_election.cc:302] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdf783afaea649e69dbca7a15145609a; no voters:
I20260808 10:03:44.747933 6212 leader_election.cc:288] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:44.748205 6221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:44.748214 6212 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:44.748399 6221 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [term 1 LEADER]: Becoming Leader. State: Replica: fdf783afaea649e69dbca7a15145609a, State: Running, Role: LEADER
I20260808 10:03:44.748623 6221 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } }
I20260808 10:03:44.748988 6220 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdf783afaea649e69dbca7a15145609a. Latest consensus state: current_term: 1 leader_uuid: "fdf783afaea649e69dbca7a15145609a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } } }
I20260808 10:03:44.749078 6220 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:44.749650 6232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:44.750211 6232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:44.751380 6222 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdf783afaea649e69dbca7a15145609a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf783afaea649e69dbca7a15145609a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 38363 } } }
I20260808 10:03:44.751477 6222 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:44.751679 6232 catalog_manager.cc:1347] Generated new cluster ID: 9d0f9067eae14357a7d5243b733490f4
I20260808 10:03:44.751734 6232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:44.763244 6232 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:44.763749 6232 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:44.766692 6232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdf783afaea649e69dbca7a15145609a: Generated new TSK 0
I20260808 10:03:44.766904 6232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:44.770650 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--builtin_ntp_servers=127.24.162.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:03:44.859216 6246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:44.859436 6246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:44.859469 6246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:44.860586 6246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:44.860679 6246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:44.860724 6246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:44.862087 6246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:44.862349 6246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:44.862651 6246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:44.868546 6256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:44.868548 6251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:44.868556 6253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:44.868878 6246 server_base.cc:1034] running on GCE node
I20260808 10:03:44.869112 6246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:44.869355 6246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:44.870523 6246 hybrid_clock.cc:648] HybridClock initialized: now 1786183424870511 us; error 31 us; skew 500 ppm
I20260808 10:03:44.871876 6246 webserver.cc:460] Webserver started at http://127.24.162.129:44245/ using document root <none> and password file <none>
I20260808 10:03:44.872133 6246 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:44.872205 6246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:44.872366 6246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:44.873411 6246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "d492fbb8b4394065b416930f03748458"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "a2ce261fe1ab33dc9f0c45e889d81dad"
server_key_iv: "d2b5ff5079d0ebfe2c476988955a736f"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.873770 6246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "d492fbb8b4394065b416930f03748458"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "a2ce261fe1ab33dc9f0c45e889d81dad"
server_key_iv: "d2b5ff5079d0ebfe2c476988955a736f"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.875128 6246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:03:44.875949 6264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:44.876121 6246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:44.876216 6246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "d492fbb8b4394065b416930f03748458"
format_stamp: "Formatted at 2026-08-08 10:03:44 on dist-test-slave-nfb5"
server_key: "a2ce261fe1ab33dc9f0c45e889d81dad"
server_key_iv: "d2b5ff5079d0ebfe2c476988955a736f"
server_key_version: "encryptionkey@0"
I20260808 10:03:44.876299 6246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:44.913730 6246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:44.914079 6246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:44.914238 6246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:44.914495 6246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:44.914954 6246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:44.915021 6246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:44.915066 6246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:44.915144 6246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:44.921038 6246 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41439
I20260808 10:03:44.921121 6438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41439 every 8 connection(s)
I20260808 10:03:44.921461 6246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:44.926519 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:44.922216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:54343 (local address 127.24.162.190:38363)
0808 10:03:44.922308 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:44.922311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:44.922507 (+ 196us) server_negotiation.cc:407] Connection header received
0808 10:03:44.923040 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:44.923041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:44.923055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:44.923099 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:44.923712 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.924192 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.925138 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.925311 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.925363 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.926230 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:44.926231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:44.926231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:44.926235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:44.926246 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:44.926249 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:44.926286 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:44.926366 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:03:44.926415 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:03:44.926600 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6246
I20260808 10:03:44.926613 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:44.922177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38363 (local address 127.24.162.129:54343)
0808 10:03:44.922386 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:44.922420 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:44.922951 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:44.923148 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:44.923151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:44.923160 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:44.923631 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.923635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.924274 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.924275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.925064 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:44.925066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:44.925388 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:44.925389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:44.925501 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:44.926108 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:44.926108 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:44.926136 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:44.926157 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:44.926291 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:44.926295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:44.926298 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:44.926358 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:03:44.926434 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260808 10:03:44.926679 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:44.926900 25226 external_mini_cluster.cc:1428] Setting key 88e40c35cb8119f6b5266fc2a3f23787
I20260808 10:03:44.927174 6441 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38363
I20260808 10:03:44.927280 6441 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:44.927491 6441 heartbeater.cc:511] Master 127.24.162.190:38363 requested a full tablet report, sending...
I20260808 10:03:44.927966 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--builtin_ntp_servers=127.24.162.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:03:44.928022 6158 ts_manager.cc:194] Registered new tserver with Master: d492fbb8b4394065b416930f03748458 (127.24.162.129:41439)
I20260808 10:03:44.928643 6158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.129:54343
W20260808 10:03:45.010502 6444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.010739 6444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.010778 6444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.011924 6444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.011991 6444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.012032 6444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:45.013360 6444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.013625 6444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.013933 6444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:45.019130 6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.019146 6450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.019130 6451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.019495 6444 server_base.cc:1034] running on GCE node
I20260808 10:03:45.019686 6444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.019912 6444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.021051 6444 hybrid_clock.cc:648] HybridClock initialized: now 1786183425021040 us; error 29 us; skew 500 ppm
I20260808 10:03:45.022272 6444 webserver.cc:460] Webserver started at http://127.24.162.130:39487/ using document root <none> and password file <none>
I20260808 10:03:45.022524 6444 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.022594 6444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.022748 6444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.023888 6444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "7134c21fbb994dcd9cc8fe9b2790f845"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "5d31d47e9e9781401063b99c4fbee2ca"
server_key_iv: "c4a74fb34a568c7bbd0af579ebfbe876"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.024259 6444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "7134c21fbb994dcd9cc8fe9b2790f845"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "5d31d47e9e9781401063b99c4fbee2ca"
server_key_iv: "c4a74fb34a568c7bbd0af579ebfbe876"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.025540 6444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:45.026221 6461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.026391 6444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:45.026490 6444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "7134c21fbb994dcd9cc8fe9b2790f845"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "5d31d47e9e9781401063b99c4fbee2ca"
server_key_iv: "c4a74fb34a568c7bbd0af579ebfbe876"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.026575 6444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.037199 6444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.037467 6444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.037614 6444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.037837 6444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:45.038161 6444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:45.038218 6444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.038254 6444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:45.038281 6444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.044016 6444 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:35517
I20260808 10:03:45.044093 6622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:35517 every 8 connection(s)
I20260808 10:03:45.044425 6444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:45.049401 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.045147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:46469 (local address 127.24.162.190:38363)
0808 10:03:45.045234 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.045237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.045409 (+ 172us) server_negotiation.cc:407] Connection header received
0808 10:03:45.045873 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.045874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.045884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.045938 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.046552 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.047132 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.048062 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.048318 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.048373 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:45.049194 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.049196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.049196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.049200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.049211 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.049215 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.049256 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.049313 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.049335 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:03:45.049679 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.045104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38363 (local address 127.24.162.130:46469)
0808 10:03:45.045308 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:45.045340 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:45.045790 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:45.045986 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:45.045989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:45.045997 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:45.046477 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:45.046481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.047212 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.047212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:45.047990 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:45.047993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.048384 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.048384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:45.048490 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:45.049072 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:45.049073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:45.049100 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:45.049122 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:45.049258 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:45.049260 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:45.049262 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:03:45.049331 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:03:45.049476 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260808 10:03:45.050176 6623 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38363
I20260808 10:03:45.050267 6623 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:45.050405 6623 heartbeater.cc:511] Master 127.24.162.190:38363 requested a full tablet report, sending...
I20260808 10:03:45.050769 6158 ts_manager.cc:194] Registered new tserver with Master: 7134c21fbb994dcd9cc8fe9b2790f845 (127.24.162.130:35517)
I20260808 10:03:45.051148 6158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.130:46469
I20260808 10:03:45.053467 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6444
I20260808 10:03:45.053537 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:45.053682 25226 external_mini_cluster.cc:1428] Setting key 771bfe54b4bdab6a3a4993b66594c8e0
I20260808 10:03:45.054553 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--builtin_ntp_servers=127.24.162.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:03:45.135851 6627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.136044 6627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.136063 6627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.137291 6627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.137333 6627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.137358 6627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:45.138675 6627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:38363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.138939 6627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.139243 6627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:45.144187 6639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.144289 6627 server_base.cc:1034] running on GCE node
W20260808 10:03:45.144184 6632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.144220 6634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.144665 6627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.144917 6627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.146059 6627 hybrid_clock.cc:648] HybridClock initialized: now 1786183425146033 us; error 42 us; skew 500 ppm
I20260808 10:03:45.147320 6627 webserver.cc:460] Webserver started at http://127.24.162.131:37341/ using document root <none> and password file <none>
I20260808 10:03:45.147564 6627 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.147609 6627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.147797 6627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.148972 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "77ec8b5cb808432d9e4ccd22e4ddb695"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "daa9f5709defc1267895f252b4c50e5d"
server_key_iv: "c735f0db083288a6a949212a21707fcc"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.149335 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "77ec8b5cb808432d9e4ccd22e4ddb695"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "daa9f5709defc1267895f252b4c50e5d"
server_key_iv: "c735f0db083288a6a949212a21707fcc"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.150655 6627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 10:03:45.151435 6647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.151616 6627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.151692 6627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "77ec8b5cb808432d9e4ccd22e4ddb695"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "daa9f5709defc1267895f252b4c50e5d"
server_key_iv: "c735f0db083288a6a949212a21707fcc"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.151784 6627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.177796 6627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.178120 6627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.178279 6627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.178514 6627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:45.178865 6627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:45.178925 6627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.178993 6627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:45.179018 6627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.185042 6627 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:41801
I20260808 10:03:45.185093 6813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:41801 every 8 connection(s)
I20260808 10:03:45.185410 6627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:45.189857 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6627
I20260808 10:03:45.189963 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:45.190219 25226 external_mini_cluster.cc:1428] Setting key f083df5ab7c5eb0c52bfd8789eef2477
I20260808 10:03:45.191150 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.186219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:55147 (local address 127.24.162.190:38363)
0808 10:03:45.186322 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.186325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.186502 (+ 177us) server_negotiation.cc:407] Connection header received
0808 10:03:45.187094 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.187095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.187105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.187160 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.187844 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.188321 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.189389 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.189567 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.189617 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:45.190797 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.190798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.190798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.190804 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.190832 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.190837 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.190897 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.191042 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.191070 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:03:45.191416 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.186135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:38363 (local address 127.24.162.131:55147)
0808 10:03:45.186359 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:45.186413 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:45.187000 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:45.187219 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:45.187222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:45.187231 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:45.187768 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:45.187772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.188404 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.188405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:45.189310 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:45.189313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:45.189715 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:45.189716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:45.189897 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:45.190598 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:45.190598 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:45.190640 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:45.190677 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:45.190943 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:45.190949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:45.190952 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:45.191035 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:03:45.191145 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260808 10:03:45.191910 6814 heartbeater.cc:347] Connected to a master server at 127.24.162.190:38363
I20260808 10:03:45.191998 6814 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:45.192222 6814 heartbeater.cc:511] Master 127.24.162.190:38363 requested a full tablet report, sending...
I20260808 10:03:45.192613 6158 ts_manager.cc:194] Registered new tserver with Master: 77ec8b5cb808432d9e4ccd22e4ddb695 (127.24.162.131:41801)
I20260808 10:03:45.192989 6158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.131:55147
I20260808 10:03:45.202353 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:03:45.204432 6218 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:45.203848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.24.162.190:38363)
0808 10:03:45.203895 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.203898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.204028 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:03:45.204100 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.204101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.204112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.204182 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.204342 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58356: BlockingRecv error: recv got EOF from 127.0.0.1:58356 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:45.204522 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6246
I20260808 10:03:45.223903 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6444
I20260808 10:03:45.230686 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6627
I20260808 10:03:45.240705 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6113
2026-08-08T10:03:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (659 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T10:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:45Z Disabled control of system clock
I20260808 10:03:45.273375 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40813
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:40813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:03:45.354672 6829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.354876 6829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.354897 6829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.356041 6829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:45.356086 6829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.356099 6829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.356112 6829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:45.356124 6829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:45.357477 6829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:40813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.357676 6829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.357913 6829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:03:45.362820 6829 server_base.cc:1034] running on GCE node
W20260808 10:03:45.362975 6843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.362830 6840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.362877 6839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.363348 6829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.363637 6829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.364804 6829 hybrid_clock.cc:648] HybridClock initialized: now 1786183425364796 us; error 31 us; skew 500 ppm
I20260808 10:03:45.365952 6829 webserver.cc:460] Webserver started at http://127.24.162.190:39355/ using document root <none> and password file <none>
I20260808 10:03:45.366195 6829 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.366266 6829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.366438 6829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.367607 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "165d599b1e664827ba292087c977426f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "06b86bbdee4f4cfac71f4345e394198c"
server_key_iv: "9fef59b01c482dc58ef9dbb92e375751"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.367967 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "165d599b1e664827ba292087c977426f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "06b86bbdee4f4cfac71f4345e394198c"
server_key_iv: "9fef59b01c482dc58ef9dbb92e375751"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.369259 6829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:45.370023 6850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.370201 6829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:45.370306 6829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "165d599b1e664827ba292087c977426f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "06b86bbdee4f4cfac71f4345e394198c"
server_key_iv: "9fef59b01c482dc58ef9dbb92e375751"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.370386 6829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.373718 6829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.373987 6829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.374130 6829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.378331 6829 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:40813
I20260808 10:03:45.378362 6921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:40813 every 8 connection(s)
I20260808 10:03:45.378670 6829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:45.379419 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:45.382090 6922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: Bootstrap starting.
I20260808 10:03:45.382679 6922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:45.383060 6922 log.cc:824] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:45.383883 6922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: No bootstrap required, opened a new log
I20260808 10:03:45.385492 6922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } }
I20260808 10:03:45.385624 6922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:45.385672 6922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165d599b1e664827ba292087c977426f, State: Initialized, Role: FOLLOWER
I20260808 10:03:45.385823 6922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } }
I20260808 10:03:45.385901 6922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:45.385947 6922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:45.385998 6922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:45.386525 6922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } }
I20260808 10:03:45.386641 6922 leader_election.cc:302] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 165d599b1e664827ba292087c977426f; no voters:
I20260808 10:03:45.386835 6922 leader_election.cc:288] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:45.386965 6927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:45.387142 6927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [term 1 LEADER]: Becoming Leader. State: Replica: 165d599b1e664827ba292087c977426f, State: Running, Role: LEADER
I20260808 10:03:45.387153 6922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:45.387246 6927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } }
I20260808 10:03:45.387760 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6829
I20260808 10:03:45.387873 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:45.388108 25226 external_mini_cluster.cc:1428] Setting key 2c924197c46566d0ed35696fc9be33a6
I20260808 10:03:45.388430 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "165d599b1e664827ba292087c977426f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } } }
I20260808 10:03:45.388418 6932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 165d599b1e664827ba292087c977426f. Latest consensus state: current_term: 1 leader_uuid: "165d599b1e664827ba292087c977426f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "165d599b1e664827ba292087c977426f" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40813 } } }
I20260808 10:03:45.388550 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:45.388561 6932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f [sys.catalog]: This master's current role is: LEADER
W20260808 10:03:45.389310 6951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:45.389386 6951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:45.389451 6949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:45.389662 6949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:03:45.391131 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.388832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.24.162.190:40813)
0808 10:03:45.389532 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.389540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.389575 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:03:45.389693 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.389701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.389746 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.389891 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.390627 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.390630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.390631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.390635 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.390697 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.390711 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.390775 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.390879 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.390957 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":61,"threads_started":1}
I20260808 10:03:45.391402 6949 catalog_manager.cc:1347] Generated new cluster ID: aebc4cbd846349eeb5f25933e8130cc1
I20260808 10:03:45.391455 6949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:45.404058 6949 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:45.404237 6949 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:45.410503 6949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 165d599b1e664827ba292087c977426f: Generated new TSK 0
I20260808 10:03:45.410691 6949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:03:45.414755 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--builtin_ntp_servers=127.24.162.148:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:03:45.498219 6956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.498461 6956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.498497 6956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.499820 6956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.499884 6956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.499926 6956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:45.501313 6956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.501590 6956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.501909 6956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:45.508025 6961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.508049 6962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.508337 6956 server_base.cc:1034] running on GCE node
W20260808 10:03:45.508046 6965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.508569 6956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.508821 6956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.509986 6956 hybrid_clock.cc:648] HybridClock initialized: now 1786183425509966 us; error 38 us; skew 500 ppm
I20260808 10:03:45.511355 6956 webserver.cc:460] Webserver started at http://127.24.162.129:38477/ using document root <none> and password file <none>
I20260808 10:03:45.511610 6956 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.511684 6956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.511852 6956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.512907 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "90fb12265dff48dcaf6fd38d92f95f0b"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "0410b7aaa909ba7697dccaf1623e32d6"
server_key_iv: "f72d437ecee2b50590da1396ffc194f2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.513268 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "90fb12265dff48dcaf6fd38d92f95f0b"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "0410b7aaa909ba7697dccaf1623e32d6"
server_key_iv: "f72d437ecee2b50590da1396ffc194f2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.514655 6956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:45.515516 6977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.515729 6956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:03:45.515836 6956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "90fb12265dff48dcaf6fd38d92f95f0b"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "0410b7aaa909ba7697dccaf1623e32d6"
server_key_iv: "f72d437ecee2b50590da1396ffc194f2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.515920 6956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.519260 6956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.519519 6956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.519666 6956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.519918 6956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:45.520228 6956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:45.520295 6956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.520336 6956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:45.520364 6956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.526286 6956 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:37451
I20260808 10:03:45.526348 7126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:37451 every 8 connection(s)
I20260808 10:03:45.526643 6956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:03:45.529629 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.527425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59015 (local address 127.24.162.190:40813)
0808 10:03:45.527497 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.527499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.527695 (+ 196us) server_negotiation.cc:407] Connection header received
0808 10:03:45.528129 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.528130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.528137 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.528223 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.529394 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.529395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.529396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.529400 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.529417 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.529421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.529482 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.529557 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.529571 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:03:45.529817 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.527393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40813 (local address 127.24.162.129:59015)
0808 10:03:45.527570 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:45.527611 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:45.528048 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:45.528274 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:45.528279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:45.528287 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:45.529245 (+ 958us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:45.529246 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:45.529275 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:45.529303 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:45.529481 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:45.529483 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:45.529486 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:45.529554 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:03:45.529614 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260808 10:03:45.530213 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 6956
I20260808 10:03:45.530336 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:45.530445 7127 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40813
I20260808 10:03:45.530539 7127 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:45.530560 25226 external_mini_cluster.cc:1428] Setting key 2e3a9d808323905cbdf6e0db481418fc
I20260808 10:03:45.530694 7127 heartbeater.cc:511] Master 127.24.162.190:40813 requested a full tablet report, sending...
I20260808 10:03:45.531311 6877 ts_manager.cc:194] Registered new tserver with Master: 90fb12265dff48dcaf6fd38d92f95f0b (127.24.162.129:37451)
I20260808 10:03:45.531626 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--builtin_ntp_servers=127.24.162.148:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:03:45.613099 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.613292 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.613314 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.614446 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.614485 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.614511 7133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:45.615933 7133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.616261 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.616585 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:45.621753 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.621712 7144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.621762 7133 server_base.cc:1034] running on GCE node
W20260808 10:03:45.621726 7142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.622180 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.622442 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.623579 7133 hybrid_clock.cc:648] HybridClock initialized: now 1786183425623560 us; error 41 us; skew 500 ppm
I20260808 10:03:45.624897 7133 webserver.cc:460] Webserver started at http://127.24.162.130:44843/ using document root <none> and password file <none>
I20260808 10:03:45.625156 7133 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.625205 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.625376 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.626406 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "7c84524d09104f848da1cfa3eda7ca7f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "05245474e36af100370de5043ed3d83c"
server_key_iv: "d99cf7c2a41eebb11c843319346c8bc2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.626809 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "7c84524d09104f848da1cfa3eda7ca7f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "05245474e36af100370de5043ed3d83c"
server_key_iv: "d99cf7c2a41eebb11c843319346c8bc2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.628168 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:03:45.628895 7153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.629065 7133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:03:45.629170 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "7c84524d09104f848da1cfa3eda7ca7f"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "05245474e36af100370de5043ed3d83c"
server_key_iv: "d99cf7c2a41eebb11c843319346c8bc2"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.629248 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.632615 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.632869 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.633021 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.633226 7133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:45.633525 7133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:45.633580 7133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.633618 7133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:45.633647 7133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.639448 7133 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:43833
I20260808 10:03:45.639525 7312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:43833 every 8 connection(s)
I20260808 10:03:45.639845 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:45.642485 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.640501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:59503 (local address 127.24.162.190:40813)
0808 10:03:45.640580 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.640583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.640773 (+ 190us) server_negotiation.cc:407] Connection header received
0808 10:03:45.641165 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.641166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.641172 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.641222 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.642265 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.642266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.642267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.642270 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.642286 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.642290 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.642342 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.642420 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.642432 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:03:45.642644 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.640446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40813 (local address 127.24.162.130:59503)
0808 10:03:45.640655 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:45.640688 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:45.641090 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:45.641276 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:45.641279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:45.641287 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:45.642120 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:45.642120 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:45.642145 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:45.642171 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:45.642351 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:45.642356 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:45.642358 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:03:45.642416 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:03:45.642462 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260808 10:03:45.643213 7313 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40813
I20260808 10:03:45.643287 7313 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:45.643388 7313 heartbeater.cc:511] Master 127.24.162.190:40813 requested a full tablet report, sending...
I20260808 10:03:45.643692 6877 ts_manager.cc:194] Registered new tserver with Master: 7c84524d09104f848da1cfa3eda7ca7f (127.24.162.130:43833)
I20260808 10:03:45.646889 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 7133
I20260808 10:03:45.647046 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:45.647243 25226 external_mini_cluster.cc:1428] Setting key 2f0e7e5ec940db2a1d27cf2e14f9f216
I20260808 10:03:45.648216 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--builtin_ntp_servers=127.24.162.148:40741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:03:45.729099 7317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:45.729292 7317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:45.729314 7317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:45.730497 7317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:45.730538 7317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:45.730563 7317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:45.732002 7317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:45.732266 7317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:45.732571 7317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:45.737818 7324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:45.737852 7329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.737835 7317 server_base.cc:1034] running on GCE node
W20260808 10:03:45.737818 7322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:45.738327 7317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:45.738586 7317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:45.739748 7317 hybrid_clock.cc:648] HybridClock initialized: now 1786183425739713 us; error 51 us; skew 500 ppm
I20260808 10:03:45.741168 7317 webserver.cc:460] Webserver started at http://127.24.162.131:36501/ using document root <none> and password file <none>
I20260808 10:03:45.741453 7317 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:45.741528 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:45.741695 7317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:45.742761 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "53261fa840844380bf607f2401b43a73"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "b2002f063f1d686d6338ae6b8632e5e9"
server_key_iv: "5a52439b83adf8839a78dbf9e7c4a93f"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.743206 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "53261fa840844380bf607f2401b43a73"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "b2002f063f1d686d6338ae6b8632e5e9"
server_key_iv: "5a52439b83adf8839a78dbf9e7c4a93f"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.744553 7317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:45.745350 7340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.745550 7317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:45.745661 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "53261fa840844380bf607f2401b43a73"
format_stamp: "Formatted at 2026-08-08 10:03:45 on dist-test-slave-nfb5"
server_key: "b2002f063f1d686d6338ae6b8632e5e9"
server_key_iv: "5a52439b83adf8839a78dbf9e7c4a93f"
server_key_version: "encryptionkey@0"
I20260808 10:03:45.745738 7317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:45.749154 7317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:45.749414 7317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:45.749562 7317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:45.749776 7317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:45.750093 7317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:45.750154 7317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.750193 7317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:45.750222 7317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:45.757085 7317 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:39955
I20260808 10:03:45.757138 7483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:39955 every 8 connection(s)
I20260808 10:03:45.757462 7317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:45.760488 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.758229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:55507 (local address 127.24.162.190:40813)
0808 10:03:45.758296 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.758298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.758516 (+ 218us) server_negotiation.cc:407] Connection header received
0808 10:03:45.759045 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.759046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.759051 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.759118 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.760303 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:45.760304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:45.760305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:03:45.760308 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:45.760321 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:03:45.760325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:45.760374 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:45.760420 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 10:03:45.760433 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:03:45.760766 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:45.758211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40813 (local address 127.24.162.131:55507)
0808 10:03:45.758386 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:45.758428 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:45.758882 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:45.759172 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:45.759175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:45.759184 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:45.760156 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:03:45.760156 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:45.760186 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:03:45.760213 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:45.760373 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:45.760375 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:45.760378 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:45.760428 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 10:03:45.760477 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 10:03:45.761317 7484 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40813
I20260808 10:03:45.761399 7484 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:45.761510 7484 heartbeater.cc:511] Master 127.24.162.190:40813 requested a full tablet report, sending...
I20260808 10:03:45.761852 6877 ts_manager.cc:194] Registered new tserver with Master: 53261fa840844380bf607f2401b43a73 (127.24.162.131:39955)
I20260808 10:03:45.763706 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 7317
I20260808 10:03:45.763795 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:45.763963 25226 external_mini_cluster.cc:1428] Setting key 982a052c1537424749128441ac18cfc3
I20260808 10:03:45.765147 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:03:45.766997 6952 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:03:45.766492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.24.162.190:40813)
0808 10:03:45.766537 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:45.766539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:45.766625 (+ 86us) server_negotiation.cc:407] Connection header received
0808 10:03:45.766661 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:45.766661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:45.766665 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:45.766727 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:45.766852 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53984: BlockingRecv error: recv got EOF from 127.0.0.1:53984 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:03:45.767040 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6956
I20260808 10:03:45.777900 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 7133
I20260808 10:03:45.788291 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 7317
I20260808 10:03:45.797865 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 6829
2026-08-08T10:03:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (558 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:45 dist-test-slave-nfb5 krb5kdc[7501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:45 dist-test-slave-nfb5 krb5kdc[7501](info): set up 2 sockets
Aug 08 10:03:45 dist-test-slave-nfb5 krb5kdc[7501](info): commencing operation
krb5kdc: starting...
W20260808 10:03:47.036276 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:03:47.159185 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40467
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:40467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:47.240579 7516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:47.240804 7516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:47.240829 7516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:47.242048 7516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:03:47.242092 7516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:47.242106 7516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:47.242120 7516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:03:47.242132 7516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:03:47.243602 7516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:40467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:47.244050 7516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:47.244244 7516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:47.249071 7526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.249070 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.249070 7530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:47.249361 7516 server_base.cc:1034] running on GCE node
I20260808 10:03:47.249697 7516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:47.249989 7516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:47.251152 7516 hybrid_clock.cc:648] HybridClock initialized: now 1786183427251155 us; error 47 us; skew 500 ppm
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:47.253892 7516 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:03:47.254374 7516 webserver.cc:460] Webserver started at http://127.24.162.190:40497/ using document root <none> and password file <none>
I20260808 10:03:47.254604 7516 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:47.254675 7516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:47.254840 7516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:47.255927 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "34c24f2fdc184b29b8ca021f145d2fd1"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "5168085cdda359dc5d5d0bf7b4f055e3"
server_key_iv: "0f147774da21f75723810ae3069ed58f"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.256301 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "34c24f2fdc184b29b8ca021f145d2fd1"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "5168085cdda359dc5d5d0bf7b4f055e3"
server_key_iv: "0f147774da21f75723810ae3069ed58f"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.257622 7516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:47.258340 7540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.258518 7516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:47.258626 7516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "34c24f2fdc184b29b8ca021f145d2fd1"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "5168085cdda359dc5d5d0bf7b4f055e3"
server_key_iv: "0f147774da21f75723810ae3069ed58f"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.258702 7516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:47.281275 7516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:47.283787 7516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:47.283954 7516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:47.288184 7516 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:40467
I20260808 10:03:47.288244 7613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:40467 every 8 connection(s)
I20260808 10:03:47.288618 7516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:03:47.289637 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:47.292047 7615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: Bootstrap starting.
I20260808 10:03:47.292645 7615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:47.292965 7615 log.cc:824] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:47.293732 7615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: No bootstrap required, opened a new log
I20260808 10:03:47.293818 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 7516
I20260808 10:03:47.293946 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:03:47.294145 25226 external_mini_cluster.cc:1428] Setting key 7b422276f78973f6777721dd9eda7fc9
I20260808 10:03:47.296290 7615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } }
I20260808 10:03:47.296471 7615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:47.296518 7615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c24f2fdc184b29b8ca021f145d2fd1, State: Initialized, Role: FOLLOWER
I20260808 10:03:47.296658 7615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } }
I20260808 10:03:47.296757 7615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:03:47.296797 7615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:03:47.296847 7615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:47.297505 7615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } }
I20260808 10:03:47.297598 7615 leader_election.cc:302] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c24f2fdc184b29b8ca021f145d2fd1; no voters:
I20260808 10:03:47.297821 7615 leader_election.cc:288] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:03:47.297854 7623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:47.297963 7623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 34c24f2fdc184b29b8ca021f145d2fd1, State: Running, Role: LEADER
I20260808 10:03:47.298059 7623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } }
I20260808 10:03:47.298097 7615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:03:47.300056 7624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } } }
I20260808 10:03:47.300266 7626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c24f2fdc184b29b8ca021f145d2fd1. Latest consensus state: current_term: 1 leader_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c24f2fdc184b29b8ca021f145d2fd1" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40467 } } }
I20260808 10:03:47.300474 7624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [sys.catalog]: This master's current role is: LEADER
I20260808 10:03:47.300480 7626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1 [sys.catalog]: This master's current role is: LEADER
W20260808 10:03:47.300940 7642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:03:47.301033 7642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:03:47.301117 7644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:03:47.301335 7644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:47.303049 7644 catalog_manager.cc:1347] Generated new cluster ID: 80421fb3b78545e58138e2484c68f0c3
I20260808 10:03:47.303108 7644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:03:47.306643 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.294705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.24.162.190:40467)
0808 10:03:47.294897 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.294905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.294992 (+ 87us) server_negotiation.cc:407] Connection header received
0808 10:03:47.296224 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.296231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.296280 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.296397 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:47.297234 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.298429 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.299562 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.300111 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.300230 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.303384 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:47.303388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:47.303388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:47.303400 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:47.305089 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.305483 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.305486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.305487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.305576 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.305797 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.305798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.305799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.306025 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:47.306200 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.306311 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.306452 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260808 10:03:47.317338 7644 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:03:47.318022 7644 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:03:47.324142 7644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: Generated new TSK 0
I20260808 10:03:47.324344 7644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:03:47.354399 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:47.436475 7653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:47.436664 7653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:47.436686 7653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:47.437790 7653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:47.437835 7653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:47.437861 7653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:03:47.439493 7653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:47.439738 7653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:47.439982 7653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:47.445150 7659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.445160 7660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.445204 7662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:47.445364 7653 server_base.cc:1034] running on GCE node
I20260808 10:03:47.445780 7653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:47.446044 7653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:47.447201 7653 hybrid_clock.cc:648] HybridClock initialized: now 1786183427447195 us; error 36 us; skew 500 ppm
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:47.449630 7653 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:03:47.450068 7653 webserver.cc:460] Webserver started at http://127.24.162.129:42503/ using document root <none> and password file <none>
I20260808 10:03:47.450304 7653 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:47.450378 7653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:47.450534 7653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:47.451653 7653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "8e78e3fe262b76d624e8a3ab69c9d5d9"
server_key_iv: "05a96665b3b6e0aaeffe97cad08de7ec"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.452020 7653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "8e78e3fe262b76d624e8a3ab69c9d5d9"
server_key_iv: "05a96665b3b6e0aaeffe97cad08de7ec"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.453372 7653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:03:47.454140 7672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.454339 7653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:47.454447 7653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "8e78e3fe262b76d624e8a3ab69c9d5d9"
server_key_iv: "05a96665b3b6e0aaeffe97cad08de7ec"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.454519 7653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:47.461107 7653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:47.463027 7653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:47.463174 7653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:47.463387 7653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:47.463681 7653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:47.463735 7653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.463775 7653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:47.463802 7653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.469619 7653 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41269
I20260808 10:03:47.469694 7847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41269 every 8 connection(s)
I20260808 10:03:47.470016 7653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:47.479251 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.470759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:54359 (local address 127.24.162.190:40467)
0808 10:03:47.470809 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.470812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.471148 (+ 336us) server_negotiation.cc:407] Connection header received
0808 10:03:47.471624 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.471625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.471636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.471691 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:47.472836 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.473354 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.474337 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.474535 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.474590 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.476741 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:47.476743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:47.476743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:47.476750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:47.477982 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.478384 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.478385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.478385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.478436 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.478652 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.478653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.478653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.478758 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:47.478811 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.479102 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.479177 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:03:47.479446 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.470792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40467 (local address 127.24.162.129:54359)
0808 10:03:47.471027 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.471063 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.471523 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.471911 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.471914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.472517 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:47.472702 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.472707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.473439 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.473440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.474256 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.474258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.474623 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.474624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.474763 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.475258 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:47.475266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:47.476647 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:47.478076 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.478083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.478085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.478315 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.478497 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.478498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.478498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.478597 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.478815 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:47.478820 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:47.478997 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.479085 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.479208 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":182,"threads_started":1}
I20260808 10:03:47.479526 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 7653
I20260808 10:03:47.479614 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:03:47.479897 25226 external_mini_cluster.cc:1428] Setting key a452c9d40c015cfc0ec2898143e3fff3
I20260808 10:03:47.480173 7848 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40467
I20260808 10:03:47.480284 7848 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:47.480507 7848 heartbeater.cc:511] Master 127.24.162.190:40467 requested a full tablet report, sending...
I20260808 10:03:47.481026 7565 ts_manager.cc:194] Registered new tserver with Master: c538e097ff9b4e6eaf9f2a5b4490f21c (127.24.162.129:41269)
I20260808 10:03:47.481647 7565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:54359
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:03:47.507622 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:47.589943 7854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:47.590137 7854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:47.590160 7854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:47.591238 7854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:47.591272 7854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:47.591297 7854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:03:47.592664 7854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:47.592854 7854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:47.593036 7854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:47.597916 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.597949 7860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.597916 7863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:47.598003 7854 server_base.cc:1034] running on GCE node
I20260808 10:03:47.598428 7854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:47.598673 7854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:47.599838 7854 hybrid_clock.cc:648] HybridClock initialized: now 1786183427599815 us; error 67 us; skew 500 ppm
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:47.602466 7854 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:03:47.602906 7854 webserver.cc:460] Webserver started at http://127.24.162.130:42763/ using document root <none> and password file <none>
I20260808 10:03:47.603195 7854 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:47.603266 7854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:47.603432 7854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:47.604482 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "c8a38efd92144044ae46e80f9617d46b"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "f0cf1c108f1ee30c07c20552f6846fa2"
server_key_iv: "c0be0db43784d456a80b400c40372d87"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.604856 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "c8a38efd92144044ae46e80f9617d46b"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "f0cf1c108f1ee30c07c20552f6846fa2"
server_key_iv: "c0be0db43784d456a80b400c40372d87"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.606168 7854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:47.606891 7871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.607103 7854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:03:47.607203 7854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "c8a38efd92144044ae46e80f9617d46b"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "f0cf1c108f1ee30c07c20552f6846fa2"
server_key_iv: "c0be0db43784d456a80b400c40372d87"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.607283 7854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:47.627393 7854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:47.629347 7854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:47.629513 7854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:47.629747 7854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:47.630082 7854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:47.630147 7854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.630191 7854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:47.630234 7854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.636490 7854 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:35957
I20260808 10:03:47.636574 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:35957 every 8 connection(s)
I20260808 10:03:47.637006 7854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:03:47.644197 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 7854
I20260808 10:03:47.644332 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:03:47.644583 25226 external_mini_cluster.cc:1428] Setting key dae5363aa534c9262de82f78dcae4588
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:47.653056 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.637922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:55703 (local address 127.24.162.190:40467)
0808 10:03:47.637997 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.637999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.638204 (+ 205us) server_negotiation.cc:407] Connection header received
0808 10:03:47.639380 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.639382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.639398 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.639488 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:47.640388 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.641209 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.645087 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.645405 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.645482 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.649465 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:47.649466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:47.649467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:47.649476 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:47.651416 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.651961 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.651962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.651963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.652033 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.652292 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.652293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.652294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.652457 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:47.652531 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.652835 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.652928 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:03:47.653180 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.637889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40467 (local address 127.24.162.130:55703)
0808 10:03:47.638080 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.638124 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.639216 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.639561 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.639565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.639997 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:47.640272 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.640277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.641334 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.641336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.644994 (+ 3658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.644998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.647016 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.647018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.647185 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.647807 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:47.647817 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:47.649368 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:47.651512 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.651518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.651520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.651860 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.652101 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.652102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.652102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.652211 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.652522 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:47.652529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:47.652665 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.652758 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.652970 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260808 10:03:47.653887 8028 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40467
I20260808 10:03:47.653986 8028 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:47.654188 8028 heartbeater.cc:511] Master 127.24.162.190:40467 requested a full tablet report, sending...
I20260808 10:03:47.654603 7565 ts_manager.cc:194] Registered new tserver with Master: c8a38efd92144044ae46e80f9617d46b (127.24.162.130:35957)
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
I20260808 10:03:47.655160 7565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:55703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:03:47.673403 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:39571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:03:47.755323 8035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:03:47.755594 8035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:03:47.755638 8035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:03:47.756826 8035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:03:47.756902 8035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:03:47.756949 8035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:03:47.758505 8035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:39571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:40467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:03:47.758795 8035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:03:47.759120 8035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:03:47.765398 8041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.765369 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:03:47.765419 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:03:47.765507 8035 server_base.cc:1034] running on GCE node
I20260808 10:03:47.765789 8035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:03:47.766084 8035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:03:47.767287 8035 hybrid_clock.cc:648] HybridClock initialized: now 1786183427767260 us; error 67 us; skew 500 ppm
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:03:47.770041 8035 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:03:47.770535 8035 webserver.cc:460] Webserver started at http://127.24.162.131:41281/ using document root <none> and password file <none>
I20260808 10:03:47.770776 8035 fs_manager.cc:359] Metadata directory not provided
I20260808 10:03:47.770855 8035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:03:47.771049 8035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:03:47.772161 8035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "dc2e9e61a2214b7eae496b3e14d564bb"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "1e5e961802afd8430bc7aa24f2032604"
server_key_iv: "95c7b7b0dced1da4aa14f0c8e416e322"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.772575 8035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "dc2e9e61a2214b7eae496b3e14d564bb"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "1e5e961802afd8430bc7aa24f2032604"
server_key_iv: "95c7b7b0dced1da4aa14f0c8e416e322"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.773950 8035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:03:47.774708 8054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.774947 8035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:03:47.775061 8035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "dc2e9e61a2214b7eae496b3e14d564bb"
format_stamp: "Formatted at 2026-08-08 10:03:47 on dist-test-slave-nfb5"
server_key: "1e5e961802afd8430bc7aa24f2032604"
server_key_iv: "95c7b7b0dced1da4aa14f0c8e416e322"
server_key_version: "encryptionkey@0"
I20260808 10:03:47.775143 8035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:03:47.786054 8035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:03:47.788187 8035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:03:47.788355 8035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:03:47.788609 8035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:03:47.788946 8035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:03:47.789003 8035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.789045 8035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:03:47.789072 8035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:03:47.795111 8035 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:35957
I20260808 10:03:47.795159 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:35957 every 8 connection(s)
I20260808 10:03:47.795471 8035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:03:47.798879 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 8035
I20260808 10:03:47.799002 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:03:47.799221 25226 external_mini_cluster.cc:1428] Setting key 3474bc322885f26921ed800ed8290c2e
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[7501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:03:47.805747 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.796208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:35185 (local address 127.24.162.190:40467)
0808 10:03:47.796274 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.796276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.796479 (+ 203us) server_negotiation.cc:407] Connection header received
0808 10:03:47.796943 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.796944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.796954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.797018 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:47.797973 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.798543 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.799914 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.800242 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.800316 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.802581 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:47.802582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:47.802582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:47.802590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:47.804122 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.804667 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.804669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.804669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.804745 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.805065 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.805066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.805066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.805225 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:47.805302 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.805614 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.805680 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:03:47.805891 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.796182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:40467 (local address 127.24.162.131:35185)
0808 10:03:47.796348 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.796389 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.796863 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.797071 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.797074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.797640 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:03:47.797888 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.797893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.798619 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.798620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.799813 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.799834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.800321 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.800323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.800477 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.801036 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:03:47.801052 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:03:47.802464 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:03:47.804200 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.804206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.804208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.804575 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.804832 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:03:47.804833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:03:47.804833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:03:47.804989 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:03:47.805292 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:03:47.805312 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:03:47.805491 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.805580 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.805699 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260808 10:03:47.806360 8208 heartbeater.cc:347] Connected to a master server at 127.24.162.190:40467
I20260808 10:03:47.806465 8208 heartbeater.cc:464] Registering TS with master...
I20260808 10:03:47.806676 8208 heartbeater.cc:511] Master 127.24.162.190:40467 requested a full tablet report, sending...
I20260808 10:03:47.807083 7565 ts_manager.cc:194] Registered new tserver with Master: dc2e9e61a2214b7eae496b3e14d564bb (127.24.162.131:35957)
I20260808 10:03:47.807456 7565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:35185
I20260808 10:03:47.811157 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:03:47.818104 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.812427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.24.162.190:40467)
0808 10:03:47.812490 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.812492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.812592 (+ 100us) server_negotiation.cc:407] Connection header received
0808 10:03:47.812626 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.812627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.812635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.812684 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:03:47.813198 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.813657 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.814298 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.814477 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.814527 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.815880 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:03:47.815881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:03:47.815881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:03:47.815887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:03:47.817063 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.817380 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.817381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.817381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.817431 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:03:47.817594 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:03:47.817595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:03:47.817595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:03:47.817752 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:03:47.817833 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.817979 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.818044 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:03:47.819667 7565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:03:47.821269 7565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:03:47.827692 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:35957 (local address 127.0.0.1:39394)
0808 10:03:47.823875 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.823885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.823968 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.824199 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.824200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.824202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:47.824246 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.824248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825539 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.825540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.826533 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.826535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.827437 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.827438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.827495 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.827505 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.827559 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.827617 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":29,"thread_start_us":48,"threads_started":1}
I20260808 10:03:47.827726 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.24.162.131:35957)
0808 10:03:47.823833 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.823839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.823956 (+ 117us) server_negotiation.cc:407] Connection header received
0808 10:03:47.824034 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.824036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.824051 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.824119 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:47.824324 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825419 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.826621 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.827354 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.827439 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.827493 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.827587 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.827633 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260808 10:03:47.828076 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.24.162.129:41269)
0808 10:03:47.824012 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.824016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.824025 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:47.824247 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.824250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.824266 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.824409 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:47.824637 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825295 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.826822 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.827557 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.827629 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.827677 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.827946 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.827988 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":91,"threads_started":1}
I20260808 10:03:47.827883 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41269 (local address 127.0.0.1:58764)
0808 10:03:47.823497 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.823526 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.823608 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.824425 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.824426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.824429 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:47.824483 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.824485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825397 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.825398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.826728 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.826731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.827637 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.827638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.827696 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.827700 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.827755 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.827806 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":63,"thread_start_us":74,"threads_started":1}
I20260808 10:03:47.828644 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:35957 (local address 127.0.0.1:54642)
0808 10:03:47.823813 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.823840 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.823892 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.824100 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.824103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.824105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:47.824169 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.824171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825529 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.825530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.826730 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.826731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.828413 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.828414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.828489 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.828493 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.828526 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.828568 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":25,"thread_start_us":52,"threads_started":1}
I20260808 10:03:47.828828 8121 tablet_service.cc:1467] Processing CreateTablet for tablet a6300d8085c948529ee1474612eb4238 (DEFAULT_TABLE table=test-table [id=79f900ca5d5a4135b4533828638f3a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:47.828855 7757 tablet_service.cc:1467] Processing CreateTablet for tablet a6300d8085c948529ee1474612eb4238 (DEFAULT_TABLE table=test-table [id=79f900ca5d5a4135b4533828638f3a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:47.828980 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.823691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.24.162.130:35957)
0808 10:03:47.823895 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.823901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.823912 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:47.823967 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.823969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.823986 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.824093 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:47.824261 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.825413 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.827715 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.828303 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.828781 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.828846 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.828872 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.828913 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260808 10:03:47.829197 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6300d8085c948529ee1474612eb4238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:47.829197 8121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6300d8085c948529ee1474612eb4238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:47.832553 8230 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: Bootstrap starting.
I20260808 10:03:47.832553 8229 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Bootstrap starting.
I20260808 10:03:47.833104 7931 tablet_service.cc:1467] Processing CreateTablet for tablet a6300d8085c948529ee1474612eb4238 (DEFAULT_TABLE table=test-table [id=79f900ca5d5a4135b4533828638f3a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:03:47.833297 8230 tablet_bootstrap.cc:654] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:47.833297 8229 tablet_bootstrap.cc:654] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:47.833433 7931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6300d8085c948529ee1474612eb4238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:03:47.833954 8229 log.cc:824] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:47.833989 8230 log.cc:824] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:47.834903 8229 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: No bootstrap required, opened a new log
I20260808 10:03:47.835017 8229 ts_tablet_manager.cc:1397] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260808 10:03:47.835018 8230 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: No bootstrap required, opened a new log
I20260808 10:03:47.835081 8230 ts_tablet_manager.cc:1397] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:03:47.836338 8229 raft_consensus.cc:348] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.836493 8229 raft_consensus.cc:374] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:47.836557 8229 raft_consensus.cc:729] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc2e9e61a2214b7eae496b3e14d564bb, State: Initialized, Role: FOLLOWER
I20260808 10:03:47.836691 8229 consensus_queue.cc:260] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.836812 8233 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Bootstrap starting.
I20260808 10:03:47.837072 8208 heartbeater.cc:503] Master 127.24.162.190:40467 was elected leader, sending a full tablet report...
I20260808 10:03:47.837078 8229 ts_tablet_manager.cc:1428] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:03:47.837143 8230 raft_consensus.cc:348] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.837299 8230 raft_consensus.cc:374] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:47.837347 8230 raft_consensus.cc:729] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c538e097ff9b4e6eaf9f2a5b4490f21c, State: Initialized, Role: FOLLOWER
I20260808 10:03:47.837561 8230 consensus_queue.cc:260] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.837678 8233 tablet_bootstrap.cc:654] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Neither blocks nor log segments found. Creating new log.
I20260808 10:03:47.837865 8230 ts_tablet_manager.cc:1428] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:03:47.837955 7848 heartbeater.cc:503] Master 127.24.162.190:40467 was elected leader, sending a full tablet report...
I20260808 10:03:47.838366 8233 log.cc:824] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:03:47.839143 8233 tablet_bootstrap.cc:492] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: No bootstrap required, opened a new log
I20260808 10:03:47.839222 8233 ts_tablet_manager.cc:1397] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:47.840478 8233 raft_consensus.cc:348] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.840615 8233 raft_consensus.cc:374] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:03:47.840674 8233 raft_consensus.cc:729] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a38efd92144044ae46e80f9617d46b, State: Initialized, Role: FOLLOWER
I20260808 10:03:47.840781 8233 consensus_queue.cc:260] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.841033 8233 ts_tablet_manager.cc:1428] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:03:47.841077 8028 heartbeater.cc:503] Master 127.24.162.190:40467 was elected leader, sending a full tablet report...
I20260808 10:03:47.852784 8235 raft_consensus.cc:482] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:03:47.852862 8235 raft_consensus.cc:504] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.853116 8235 leader_election.cc:288] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8a38efd92144044ae46e80f9617d46b (127.24.162.130:35957), dc2e9e61a2214b7eae496b3e14d564bb (127.24.162.131:35957)
I20260808 10:03:47.856729 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.853263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:35957 (local address 127.24.162.129:53055)
0808 10:03:47.853308 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.853332 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.853374 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.853540 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.853541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.853543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:47.853619 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.853621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.854500 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.854500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.855718 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.855720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.856463 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.856464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.856550 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.856553 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.856611 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.856655 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":37}
I20260808 10:03:47.856911 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.853342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:47137 (local address 127.24.162.130:35957)
0808 10:03:47.853394 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.853395 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.853686 (+ 291us) server_negotiation.cc:407] Connection header received
0808 10:03:47.853777 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.853778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.853787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.853850 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:47.853945 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.854594 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.855891 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.856472 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.856542 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.856548 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.856785 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.856835 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:03:47.856923 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.853340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:53055 (local address 127.24.162.131:35957)
0808 10:03:47.853405 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.853407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.853416 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:03:47.853440 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.853441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.853450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.853509 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:03:47.853699 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.854428 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.855814 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.856388 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.856466 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.856475 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.856801 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.856866 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:03:47.857139 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.853265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:35957 (local address 127.24.162.129:47137)
0808 10:03:47.853592 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.853604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:03:47.853670 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:03:47.853854 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:03:47.853855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:03:47.853856 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:03:47.853903 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.853904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.854657 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.854657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.855811 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:03:47.855813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.856546 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.856547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:03:47.856634 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.856637 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:03:47.857010 (+ 373us) client_negotiation.cc:240] Negotiation successful
0808 10:03:47.857058 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":163,"thread_start_us":97,"threads_started":1}
I20260808 10:03:47.857246 8148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6300d8085c948529ee1474612eb4238" candidate_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" is_pre_election: true
I20260808 10:03:47.857394 8148 raft_consensus.cc:2454] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c538e097ff9b4e6eaf9f2a5b4490f21c in term 0.
I20260808 10:03:47.857453 7964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6300d8085c948529ee1474612eb4238" candidate_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a38efd92144044ae46e80f9617d46b" is_pre_election: true
I20260808 10:03:47.857594 7674 leader_election.cc:302] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c538e097ff9b4e6eaf9f2a5b4490f21c, dc2e9e61a2214b7eae496b3e14d564bb; no voters:
I20260808 10:03:47.857610 7964 raft_consensus.cc:2454] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c538e097ff9b4e6eaf9f2a5b4490f21c in term 0.
I20260808 10:03:47.857702 8235 raft_consensus.cc:2781] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:03:47.857759 8235 raft_consensus.cc:482] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:03:47.857781 8235 raft_consensus.cc:3037] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:47.858631 8235 raft_consensus.cc:504] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.858783 8235 leader_election.cc:288] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [CANDIDATE]: Term 1 election: Requested vote from peers c8a38efd92144044ae46e80f9617d46b (127.24.162.130:35957), dc2e9e61a2214b7eae496b3e14d564bb (127.24.162.131:35957)
I20260808 10:03:47.858934 7964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6300d8085c948529ee1474612eb4238" candidate_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a38efd92144044ae46e80f9617d46b"
I20260808 10:03:47.858979 8148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6300d8085c948529ee1474612eb4238" candidate_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc2e9e61a2214b7eae496b3e14d564bb"
I20260808 10:03:47.859015 7964 raft_consensus.cc:3037] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:47.859041 8148 raft_consensus.cc:3037] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:03:47.859858 8148 raft_consensus.cc:2454] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c538e097ff9b4e6eaf9f2a5b4490f21c in term 1.
I20260808 10:03:47.859855 7964 raft_consensus.cc:2454] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c538e097ff9b4e6eaf9f2a5b4490f21c in term 1.
I20260808 10:03:47.860064 7684 leader_election.cc:302] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c538e097ff9b4e6eaf9f2a5b4490f21c, c8a38efd92144044ae46e80f9617d46b; no voters:
I20260808 10:03:47.860164 8235 raft_consensus.cc:2781] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:03:47.860350 8235 raft_consensus.cc:686] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [term 1 LEADER]: Becoming Leader. State: Replica: c538e097ff9b4e6eaf9f2a5b4490f21c, State: Running, Role: LEADER
I20260808 10:03:47.860443 8235 consensus_queue.cc:237] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } }
I20260808 10:03:47.861260 7565 catalog_manager.cc:5495] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c reported cstate change: term changed from 0 to 1, leader changed from <none> to c538e097ff9b4e6eaf9f2a5b4490f21c (127.24.162.129). New cstate: current_term: 1 leader_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c538e097ff9b4e6eaf9f2a5b4490f21c" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 } health_report { overall_health: UNKNOWN } } }
I20260808 10:03:47.872012 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.869581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.24.162.129:41269)
0808 10:03:47.869625 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:03:47.869628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:03:47.869639 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:03:47.869683 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:03:47.869684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:03:47.869709 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:03:47.869780 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:03:47.869932 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.870382 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.871209 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:03:47.871403 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:03:47.871489 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:03:47.871673 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:03:47.871762 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:03:47.871868 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:03:47.871896 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:03:47.871916 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:03:47.873678 7964 raft_consensus.cc:1261] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 1 FOLLOWER]: Refusing update from remote peer c538e097ff9b4e6eaf9f2a5b4490f21c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:47.873749 8148 raft_consensus.cc:1261] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 1 FOLLOWER]: Refusing update from remote peer c538e097ff9b4e6eaf9f2a5b4490f21c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:03:47.873979 8235 consensus_queue.cc:1035] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a38efd92144044ae46e80f9617d46b" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 35957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:47.874118 8239 consensus_queue.cc:1035] T a6300d8085c948529ee1474612eb4238 P c538e097ff9b4e6eaf9f2a5b4490f21c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc2e9e61a2214b7eae496b3e14d564bb" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 35957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:03:47.878996 7564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:47.879091 7564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:03:47.879848 7564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c24f2fdc184b29b8ca021f145d2fd1: Sending DeleteTablet for 3 replicas of tablet a6300d8085c948529ee1474612eb4238
I20260808 10:03:47.880190 7931 tablet_service.cc:1514] Processing DeleteTablet for tablet a6300d8085c948529ee1474612eb4238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:54642
I20260808 10:03:47.880209 8121 tablet_service.cc:1514] Processing DeleteTablet for tablet a6300d8085c948529ee1474612eb4238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:39394
I20260808 10:03:47.880435 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 7653
I20260808 10:03:47.880554 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet a6300d8085c948529ee1474612eb4238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:03:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:58764
I20260808 10:03:47.881035 8256 tablet_replica.cc:330] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: stopping tablet replica
I20260808 10:03:47.881141 8256 raft_consensus.cc:2229] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:47.881247 8256 raft_consensus.cc:2258] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:47.881421 8255 tablet_replica.cc:330] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: stopping tablet replica
I20260808 10:03:47.881546 8255 raft_consensus.cc:2229] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:03:47.881666 8255 raft_consensus.cc:2258] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:03:47.883052 8256 ts_tablet_manager.cc:1905] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:47.883052 8255 ts_tablet_manager.cc:1905] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:03:47.884773 8255 ts_tablet_manager.cc:1918] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:47.884773 8256 ts_tablet_manager.cc:1918] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:03:47.884850 8256 log.cc:1196] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/a6300d8085c948529ee1474612eb4238
I20260808 10:03:47.884919 8255 log.cc:1196] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/a6300d8085c948529ee1474612eb4238
I20260808 10:03:47.885231 8256 ts_tablet_manager.cc:1939] T a6300d8085c948529ee1474612eb4238 P dc2e9e61a2214b7eae496b3e14d564bb: Deleting consensus metadata
I20260808 10:03:47.885265 8255 ts_tablet_manager.cc:1939] T a6300d8085c948529ee1474612eb4238 P c8a38efd92144044ae46e80f9617d46b: Deleting consensus metadata
I20260808 10:03:47.885663 7541 catalog_manager.cc:4841] TS dc2e9e61a2214b7eae496b3e14d564bb (127.24.162.131:35957): tablet a6300d8085c948529ee1474612eb4238 (table test-table [id=79f900ca5d5a4135b4533828638f3a5d]) successfully deleted
I20260808 10:03:47.885718 7544 catalog_manager.cc:4841] TS c8a38efd92144044ae46e80f9617d46b (127.24.162.130:35957): tablet a6300d8085c948529ee1474612eb4238 (table test-table [id=79f900ca5d5a4135b4533828638f3a5d]) successfully deleted
W20260808 10:03:47.890730 7541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:41269 (error 108)
I20260808 10:03:47.890982 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 7854
I20260808 10:03:47.892499 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:03:47.891123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41269 (local address 127.0.0.1:58786)
0808 10:03:47.892420 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0808 10:03:47.892459 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1275}
W20260808 10:03:47.895594 7541 catalog_manager.cc:4584] TS c538e097ff9b4e6eaf9f2a5b4490f21c (127.24.162.129:41269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6300d8085c948529ee1474612eb4238: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41269: connect: Connection refused (error 111)
I20260808 10:03:47.902273 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 8035
I20260808 10:03:47.912700 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 7516
2026-08-08T10:03:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2112 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[8262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[8262](info): set up 2 sockets
Aug 08 10:03:47 dist-test-slave-nfb5 krb5kdc[8262](info): commencing operation
krb5kdc: starting...
W20260808 10:03:49.151835 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:03:49 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:03:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:03:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangeradmin_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangeradmin_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangerlookup_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangerlookup_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
I20260808 10:03:49.320026 25226 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHQacpx/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/postgres -l logfile start
2026-08-08 10:03:51.863 UTC [8311] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:03:51.863 UTC [8311] LOG: listening on IPv4 address "127.24.162.148", port 56751
2026-08-08 10:03:51.867 UTC [8311] LOG: listening on Unix socket "/tmp/.s.PGSQL.56751"
2026-08-08 10:03:51.889 UTC [8316] LOG: database system was shut down at 2026-08-08 10:03:50 UTC
2026-08-08 10:03:51.893 UTC [8311] LOG: database system is ready to accept connections
I20260808 10:03:53.027727 25226 mini_postgres.cc:96] Postgres bound to 56751
2026-08-08 10:03:53.035 UTC [8325] FATAL: database "slave" does not exist
127.24.162.148:56751 - accepting connections
I20260808 10:03:53.036651 25226 mini_ranger.cc:161] Starting Ranger...
I20260808 10:03:53.052803 25226 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:03:53.533975 25226 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:03:53.534066 25226 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin
2026-08-08 10:03:53,850 [I] DB FLAVOR :POSTGRES
2026-08-08 10:03:53,851 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:03:53,851 [I] Checking connection..
2026-08-08 10:03:53,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:03:54,110 [I] Checking connection passed.
2026-08-08 10:03:54,110 [I] --------- Verifying version history table ---------
2026-08-08 10:03:54,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:03:54,379 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:03:54,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:03:54,619 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:03:54,619 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:03:54,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:03:54.875 UTC [8420] WARNING: there is no transaction in progress
2026-08-08 10:03:54,883 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:03:54,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:03:55,128 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:03:55,128 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:03:55,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:03:55,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:03:55,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nfb5.c.gcp-upstream.internal','N') ;"
2026-08-08 10:03:55,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:03:56,145 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:03:56,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:03:56,382 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:03:56,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:03:56,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:03:56,898 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:03:56,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:03:57.871 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:57.887 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:57.903 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:57.918 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:57.934 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:57.953 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.007 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.015 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.030 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.040 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.050 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.064 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.073 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.080 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.087 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58.093 UTC [8577] WARNING: there is no transaction in progress
2026-08-08 10:03:58,714 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:03:58,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:03:58,972 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:03:58,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:03:59,242 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:03:59,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:03:59,518 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:03:59,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:03:59,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:04:00,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nfb5.c.gcp-upstream.internal';"
2026-08-08 10:04:00,282 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:04:00,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:04:00,532 [I] DB_PATCHES have already been applied
I20260808 10:04:00.538156 25226 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHQacpx/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:04:00.538255 25226 mini_ranger.cc:193] Using host: 127.24.162.148
I20260808 10:04:00.542022 25226 mini_ranger.cc:227] Ranger admin URL: http://127.24.162.148:32987
Aug 08, 2026 10:04:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:04:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:04:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:04:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:04:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:04:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:04:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:04:01.881533 25226 mini_ranger.cc:160] Time spent starting Ranger: real 8.845s user 0.000s sys 0.011s
Aug 08, 2026 10:04:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:04:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 10:04:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:04:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:04:22 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:04:23.185045 25226 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangerkms_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/rangerkms_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.162.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.162.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:04:23 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:04:23.283788 25226 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:04:23.294344 25226 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:04:24.106418 25226 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:04:24.106495 25226 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms
I20260808 10:04:24.106510 25226 mini_ranger_kms.cc:227] Using postgres at 127.24.162.148:56751
2026-08-08 10:04:24,357 [I] DB FLAVOR :POSTGRES
2026-08-08 10:04:24,357 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:04:24,357 [I] Checking connection
2026-08-08 10:04:24,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:04:24,646 [I] connection success
2026-08-08 10:04:24,646 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:04:24,646 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:04:24,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:04:24,885 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:04:24,886 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:04:24,886 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:04:24,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:04:25,159 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:04:25.630961 25226 mini_ranger_kms.cc:310] Created kms service
I20260808 10:04:25.827662 25226 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:04:25.872061 25226 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:04:25 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:04:26.290746 25226 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:04:26.291049 25226 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:04:26.291095 25226 mini_ranger_kms.cc:240] Using host: 127.24.162.148
I20260808 10:04:26.295418 25226 mini_ranger_kms.cc:276] Ranger KMS PID: 8825
I20260808 10:04:26.295542 25226 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.162.148:54055
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:04:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:04:28 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:29.023103 25226 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.739s user 0.008s sys 0.212s
I20260808 10:04:29.023212 25226 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:04:29.023340 25226 mini_ranger_kms.cc:401] 127.24.162.148:54055/kms/v1/keys
Aug 08 10:04:29 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
Aug 08 10:04:29 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183468, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
Aug 08 10:04:31 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:04:31.841720 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33019
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33019
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:54055/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:04:31.935468 8894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:04:31.935683 8894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:04:31.935707 8894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:04:31.936843 8894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:04:31.936885 8894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:04:31.936900 8894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:04:31.936913 8894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:04:31.936926 8894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:04:31.938434 8894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.162.148:54055/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33019
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:04:31.938628 8894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:04:31.938807 8894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:04:31.944285 8904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:31.944285 8907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:31.944300 8894 server_base.cc:1034] running on GCE node
W20260808 10:04:31.944306 8905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:31.944833 8894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:04:31.945171 8894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:04:31.946343 8894 hybrid_clock.cc:648] HybridClock initialized: now 1786183471946330 us; error 30 us; skew 500 ppm
Aug 08 10:04:31 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:31.952778 8894 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:04:31.953243 8894 webserver.cc:460] Webserver started at http://127.24.162.190:41315/ using document root <none> and password file <none>
I20260808 10:04:31.953493 8894 fs_manager.cc:359] Metadata directory not provided
I20260808 10:04:31.953557 8894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:04:31.953719 8894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:04:31 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:04:32.001722 8894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "18ebb2d7e4e3408c87ac63b4397b147a"
format_stamp: "Formatted at 2026-08-08 10:04:31 on dist-test-slave-nfb5"
server_key: "09d6f63fa4454be1dbc49cb8b8e76ea1"
server_key_iv: "f8cfecd4acb84ee883f2b937d4204642"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.002190 8894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "18ebb2d7e4e3408c87ac63b4397b147a"
format_stamp: "Formatted at 2026-08-08 10:04:31 on dist-test-slave-nfb5"
server_key: "09d6f63fa4454be1dbc49cb8b8e76ea1"
server_key_iv: "f8cfecd4acb84ee883f2b937d4204642"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.004145 8894 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 10:04:32.019482 8916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:32.019788 8894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:04:32.019912 8894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "18ebb2d7e4e3408c87ac63b4397b147a"
format_stamp: "Formatted at 2026-08-08 10:04:31 on dist-test-slave-nfb5"
server_key: "09d6f63fa4454be1dbc49cb8b8e76ea1"
server_key_iv: "f8cfecd4acb84ee883f2b937d4204642"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.020018 8894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:04:32.039996 8894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:04:32.041973 8894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:04:32.042131 8894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:04:32.047256 8894 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33019
I20260808 10:04:32.047286 8986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33019 every 8 connection(s)
I20260808 10:04:32.047660 8894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:04:32.048303 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 8894
I20260808 10:04:32.048393 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:04:32.048399 8988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:04:32.048573 25226 external_mini_cluster.cc:1428] Setting key 23fcdc158e6f61cbf1eeb69292cd448b
I20260808 10:04:32.051005 8988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: Bootstrap starting.
I20260808 10:04:32.051713 8988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: Neither blocks nor log segments found. Creating new log.
I20260808 10:04:32.052075 8988 log.cc:824] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:04:32.052963 8988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: No bootstrap required, opened a new log
I20260808 10:04:32.055272 8988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } }
I20260808 10:04:32.055460 8988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:04:32.055493 8988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ebb2d7e4e3408c87ac63b4397b147a, State: Initialized, Role: FOLLOWER
I20260808 10:04:32.055649 8988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } }
I20260808 10:04:32.055733 8988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:04:32.055764 8988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:04:32.055831 8988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:04:32.056520 8988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } }
I20260808 10:04:32.056635 8988 leader_election.cc:302] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18ebb2d7e4e3408c87ac63b4397b147a; no voters:
I20260808 10:04:32.056849 8988 leader_election.cc:288] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:04:32.056877 8997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:04:32.057101 8997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [term 1 LEADER]: Becoming Leader. State: Replica: 18ebb2d7e4e3408c87ac63b4397b147a, State: Running, Role: LEADER
I20260808 10:04:32.057209 8988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:04:32.057243 8997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } }
I20260808 10:04:32.057639 8997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } } }
I20260808 10:04:32.057749 8999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18ebb2d7e4e3408c87ac63b4397b147a. Latest consensus state: current_term: 1 leader_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ebb2d7e4e3408c87ac63b4397b147a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33019 } } }
I20260808 10:04:32.057781 8997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [sys.catalog]: This master's current role is: LEADER
I20260808 10:04:32.057922 8999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a [sys.catalog]: This master's current role is: LEADER
I20260808 10:04:32.058092 9002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 10:04:32 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:04:32.059161 9002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:04:32.059824 8988 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:04:32.063333 9002 catalog_manager.cc:1347] Generated new cluster ID: 83d6bf12adf54e65bf5300d8ec9c83fa
I20260808 10:04:32.063550 9002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:04:32.069037 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:32.049127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.24.162.190:33019)
0808 10:04:32.051042 (+ 1915us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:32.051052 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:32.051116 (+ 64us) server_negotiation.cc:407] Connection header received
0808 10:04:32.051726 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:32.051735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:32.051800 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:32.052003 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:04:32.052783 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.054194 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.055325 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.055915 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.056022 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:32.058935 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:04:32.058940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:04:32.058941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:04:32.058952 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:04:32.064000 (+ 5048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:32.067536 (+ 3536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:32.067539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:32.067539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:32.067639 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:32.067829 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:32.067829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:32.067830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:32.068121 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:04:32.068266 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:32.068677 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 10:04:32.068820 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1865,"thread_start_us":78,"threads_started":1}
I20260808 10:04:32.074414 9002 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:04:32.075240 9002 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:04:32.084566 9002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: Generated new TSK 0
I20260808 10:04:32.084743 9002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:04:32.761574 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:54055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:04:32.861271 9042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:04:32.861506 9042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:04:32.861548 9042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:04:32.862699 9042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:04:32.862763 9042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:04:32.862808 9042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:04:32.864326 9042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.162.148:54055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:04:32.864589 9042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:04:32.864825 9042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:04:32.871001 9053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:32.871009 9048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:32.871021 9049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:32.871563 9042 server_base.cc:1034] running on GCE node
I20260808 10:04:32.871762 9042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:04:32.872028 9042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:04:32.873178 9042 hybrid_clock.cc:648] HybridClock initialized: now 1786183472873178 us; error 39 us; skew 500 ppm
Aug 08 10:04:32 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183472, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:32.875921 9042 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:04:32.876475 9042 webserver.cc:460] Webserver started at http://127.24.162.129:38897/ using document root <none> and password file <none>
I20260808 10:04:32.876750 9042 fs_manager.cc:359] Metadata directory not provided
I20260808 10:04:32.876796 9042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:04:32.876929 9042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:04:32 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183472, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:04:32.891820 9042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "fadf4d98cf56487183809fdc594b2e53"
format_stamp: "Formatted at 2026-08-08 10:04:32 on dist-test-slave-nfb5"
server_key: "d3158188b74816db833392ce87b0cf04"
server_key_iv: "b5a51b27390024fefbcacb13df523b95"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.892593 9042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "fadf4d98cf56487183809fdc594b2e53"
format_stamp: "Formatted at 2026-08-08 10:04:32 on dist-test-slave-nfb5"
server_key: "d3158188b74816db833392ce87b0cf04"
server_key_iv: "b5a51b27390024fefbcacb13df523b95"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.896483 9042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:04:32.910727 9060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:32.910951 9042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:04:32.911052 9042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "fadf4d98cf56487183809fdc594b2e53"
format_stamp: "Formatted at 2026-08-08 10:04:32 on dist-test-slave-nfb5"
server_key: "d3158188b74816db833392ce87b0cf04"
server_key_iv: "b5a51b27390024fefbcacb13df523b95"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:32.911166 9042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:04:32.921667 9042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:04:32.924332 9042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:04:32.924546 9042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:04:32.924889 9042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:04:32.925325 9042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:04:32.925417 9042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:32.925518 9042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:04:32.925595 9042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:32.933411 9042 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:42811
I20260808 10:04:32.933928 9042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:04:32.937119 9223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:42811 every 8 connection(s)
I20260808 10:04:32.939152 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 9042
I20260808 10:04:32.939268 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:04:32.939477 25226 external_mini_cluster.cc:1428] Setting key f93faba29d623cf1a919b8e4ad9ae52e
Aug 08 10:04:32 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183472, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:04:32.949674 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:32.937407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33019 (local address 127.24.162.129:59389)
0808 10:04:32.937604 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:32.937651 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:32.938305 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:32.938790 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:32.938795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:32.939139 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:04:32.939870 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:32.939878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.940889 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.940891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:32.942187 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:32.942190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.942724 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.942725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:32.942873 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:32.943556 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:04:32.943567 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:04:32.945314 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:04:32.947753 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:32.947760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:32.947763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:32.948120 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:32.948408 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:32.948409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:32.948410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:32.948561 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:32.948928 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:04:32.948935 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:04:32.949113 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 10:04:32.949223 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:04:32.949393 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260808 10:04:32.950572 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:32.937775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59389 (local address 127.24.162.190:33019)
0808 10:04:32.938519 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:32.938521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:32.938529 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:04:32.938557 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:32.938557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:32.938570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:32.938657 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:04:32.939974 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.940767 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.942299 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:32.942590 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:32.943156 (+ 566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:32.945428 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:04:32.945430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:04:32.945430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:04:32.945439 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:04:32.947615 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:32.948241 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:32.948242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:32.948242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:32.948318 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:32.948672 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:32.948673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:32.948673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:32.948828 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:04:32.950314 (+ 1486us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:32.950365 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:04:32.950464 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":86,"threads_started":1}
I20260808 10:04:32.951231 9224 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33019
I20260808 10:04:32.951339 9224 heartbeater.cc:464] Registering TS with master...
I20260808 10:04:32.951539 9224 heartbeater.cc:511] Master 127.24.162.190:33019 requested a full tablet report, sending...
I20260808 10:04:32.952194 8938 ts_manager.cc:194] Registered new tserver with Master: fadf4d98cf56487183809fdc594b2e53 (127.24.162.129:42811)
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
I20260808 10:04:32.953184 8938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:59389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:04:32.983765 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:54055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:04:33.112913 9236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:04:33.113300 9236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:04:33.113391 9236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:04:33.115264 9236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:04:33.115484 9236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:04:33.115578 9236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:04:33.117822 9236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.162.148:54055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:04:33.118312 9236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:04:33.118649 9236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:04:33.125360 9247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:33.125769 9252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:33.126951 9236 server_base.cc:1034] running on GCE node
W20260808 10:04:33.127341 9246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:33.127656 9236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:04:33.127918 9236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:04:33.129092 9236 hybrid_clock.cc:648] HybridClock initialized: now 1786183473129071 us; error 35 us; skew 500 ppm
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:33.132367 9236 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:04:33.132867 9236 webserver.cc:460] Webserver started at http://127.24.162.130:44597/ using document root <none> and password file <none>
I20260808 10:04:33.133116 9236 fs_manager.cc:359] Metadata directory not provided
I20260808 10:04:33.133164 9236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:04:33.133301 9236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:04:33.151021 9236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "a5ade997c27742a68cfdfceae4ae9109"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "32570e627ed9ed88c55456c283e87333"
server_key_iv: "42f6ea448bc5cf4567d143bd88dc4bfc"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.151474 9236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "a5ade997c27742a68cfdfceae4ae9109"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "32570e627ed9ed88c55456c283e87333"
server_key_iv: "42f6ea448bc5cf4567d143bd88dc4bfc"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.156195 9236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 10:04:33.176757 9264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.177021 9236 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 10:04:33.177119 9236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "a5ade997c27742a68cfdfceae4ae9109"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "32570e627ed9ed88c55456c283e87333"
server_key_iv: "42f6ea448bc5cf4567d143bd88dc4bfc"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.177227 9236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:33.197336 9236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:04:33.199925 9236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:04:33.200170 9236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:04:33.200484 9236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:04:33.200944 9236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:04:33.201036 9236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.201110 9236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:04:33.201170 9236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.209308 9236 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:46473
I20260808 10:04:33.209836 9236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:04:33.214506 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 9236
I20260808 10:04:33.214619 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:04:33.214803 25226 external_mini_cluster.cc:1428] Setting key 187d244854f3c7a2ef7e7ce8a9c25919
I20260808 10:04:33.216985 9427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:46473 every 8 connection(s)
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:04:33.230072 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.218522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:37699 (local address 127.24.162.190:33019)
0808 10:04:33.218583 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.218586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.218780 (+ 194us) server_negotiation.cc:407] Connection header received
0808 10:04:33.219507 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.219508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.219523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.219554 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:04:33.220459 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.221344 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.222965 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.223278 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.223326 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.226272 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:04:33.226274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:04:33.226275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:04:33.226282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:04:33.228229 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.228787 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.228789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.228789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.228851 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.229182 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.229183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.229184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.229350 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:04:33.229399 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.229824 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.229927 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:04:33.231070 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.218246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33019 (local address 127.24.162.130:37699)
0808 10:04:33.218633 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.218690 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.219381 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.219761 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.219766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.220010 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:04:33.220291 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.220297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.221472 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.221473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.222834 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.222837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.223424 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.223425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.223567 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.224247 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:04:33.224258 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:04:33.226141 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:04:33.228324 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:33.228331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:33.228334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:33.228679 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:33.228926 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:33.228926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:33.228927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:33.229095 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:33.229451 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:04:33.229458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:04:33.229637 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.230475 (+ 838us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.230680 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":87,"threads_started":1}
I20260808 10:04:33.232008 9429 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33019
I20260808 10:04:33.232110 9429 heartbeater.cc:464] Registering TS with master...
I20260808 10:04:33.232303 9429 heartbeater.cc:511] Master 127.24.162.190:33019 requested a full tablet report, sending...
I20260808 10:04:33.232796 8938 ts_manager.cc:194] Registered new tserver with Master: a5ade997c27742a68cfdfceae4ae9109 (127.24.162.130:46473)
I20260808 10:04:33.233294 8938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:37699
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:04:33.260349 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:54055/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:04:33.395627 9437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:04:33.395989 9437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:04:33.396081 9437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:04:33.397850 9437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:04:33.398074 9437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:04:33.398178 9437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:04:33.400395 9437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.162.148:54055/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:04:33.400884 9437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:04:33.401207 9437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:04:33.408056 9446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:33.408845 9445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:04:33.409442 9450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:04:33.409552 9437 server_base.cc:1034] running on GCE node
I20260808 10:04:33.409909 9437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:04:33.410166 9437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:04:33.411326 9437 hybrid_clock.cc:648] HybridClock initialized: now 1786183473411305 us; error 34 us; skew 500 ppm
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:04:33.414881 9437 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:04:33.415390 9437 webserver.cc:460] Webserver started at http://127.24.162.131:37649/ using document root <none> and password file <none>
I20260808 10:04:33.415648 9437 fs_manager.cc:359] Metadata directory not provided
I20260808 10:04:33.415704 9437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:04:33.415841 9437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:04:33.439920 9437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "1760d6c9d981488f9968042026589be5"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "5a45e301bd29a28177d7fc96281029e7"
server_key_iv: "21ac6c9f7d0df31016c0d6acf07d7eb7"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.440487 9437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "1760d6c9d981488f9968042026589be5"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "5a45e301bd29a28177d7fc96281029e7"
server_key_iv: "21ac6c9f7d0df31016c0d6acf07d7eb7"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.447257 9437 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260808 10:04:33.466023 9457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.466348 9437 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 10:04:33.466435 9437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "1760d6c9d981488f9968042026589be5"
format_stamp: "Formatted at 2026-08-08 10:04:33 on dist-test-slave-nfb5"
server_key: "5a45e301bd29a28177d7fc96281029e7"
server_key_iv: "21ac6c9f7d0df31016c0d6acf07d7eb7"
server_key_version: "kuduclusterkey@0"
I20260808 10:04:33.466523 9437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:04:33.479102 9437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:04:33.481263 9437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:04:33.481426 9437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:04:33.481662 9437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:04:33.482031 9437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:04:33.482106 9437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.482149 9437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:04:33.482178 9437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:04:33.491298 9437 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:34043
I20260808 10:04:33.491463 9628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:34043 every 8 connection(s)
I20260808 10:04:33.491752 9437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:04:33.496803 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 9437
I20260808 10:04:33.496901 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:04:33.497090 25226 external_mini_cluster.cc:1428] Setting key 706fc92b970388ab5dfdd6bc023a03cd
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:04:33.504078 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.492585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:34191 (local address 127.24.162.190:33019)
0808 10:04:33.492679 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.492682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.492898 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:04:33.493444 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.493446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.493458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.493526 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:04:33.494111 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.494614 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.496104 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.496486 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.496567 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.500075 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:04:33.500077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:04:33.500077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:04:33.500086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:04:33.501822 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.502325 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.502326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.502326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.502385 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.502659 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.502660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.502660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.502791 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:04:33.502836 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.503884 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.503979 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:04:33.504855 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.492542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33019 (local address 127.24.162.131:34191)
0808 10:04:33.492774 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.492817 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.493333 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.493597 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.493601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.493814 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:04:33.494009 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.494013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.494732 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.494733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.495991 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.495995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.497407 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.497408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.497566 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.498230 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:04:33.498241 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:04:33.499958 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:04:33.501925 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:33.501932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:33.501934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:33.502220 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:33.502457 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:04:33.502457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:04:33.502458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:04:33.502586 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:04:33.502885 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:04:33.502891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:04:33.503697 (+ 806us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.504449 (+ 752us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.504617 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":159,"threads_started":1}
I20260808 10:04:33.505508 9630 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33019
I20260808 10:04:33.505601 9630 heartbeater.cc:464] Registering TS with master...
I20260808 10:04:33.505784 9630 heartbeater.cc:511] Master 127.24.162.190:33019 requested a full tablet report, sending...
I20260808 10:04:33.506173 8938 ts_manager.cc:194] Registered new tserver with Master: 1760d6c9d981488f9968042026589be5 (127.24.162.131:34043)
I20260808 10:04:33.506678 8938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:34191
I20260808 10:04:33.514707 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:04:33.524662 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.516354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.24.162.190:33019)
0808 10:04:33.516398 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.516400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.516675 (+ 275us) server_negotiation.cc:407] Connection header received
0808 10:04:33.516709 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.516710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.516721 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.516794 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:04:33.517592 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.518354 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.519519 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.519812 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.519886 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.521514 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:04:33.521515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:04:33.521515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:04:33.521523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:04:33.523334 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.523766 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.523768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.523768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.523832 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:04:33.524044 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:04:33.524045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:04:33.524046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:04:33.524196 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:04:33.524323 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.524463 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.524566 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:04:33.526535 8938 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:04:33.528818 8938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:04:33.543161 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46473 (local address 127.0.0.1:49382)
0808 10:04:33.538123 (+ 5743us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.538133 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.538173 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.538453 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.538454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.538456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:04:33.538493 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.538494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.539806 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.539807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.541518 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.541521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.542815 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.542816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.542901 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.542934 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.543010 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.543070 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5727,"thread_start_us":102,"threads_started":1}
I20260808 10:04:33.543954 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.24.162.130:46473)
0808 10:04:33.533213 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.533218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.538232 (+ 5014us) server_negotiation.cc:407] Connection header received
0808 10:04:33.538288 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.538290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.538305 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.538375 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:04:33.538563 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.539704 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.541792 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.542623 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.542715 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.542775 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.543809 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.543854 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":98,"threads_started":1}
I20260808 10:04:33.545056 9325 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f (DEFAULT_TABLE table=test-table [id=1ec38f3ef6e045aaa3104df1142e7289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:04:33.545480 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbf8ed2b82648b292baad29796a5f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:04:33.547276 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:34043 (local address 127.0.0.1:36926)
0808 10:04:33.532822 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.533285 (+ 463us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.533353 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.537936 (+ 4583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.537940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.537942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:04:33.538009 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.538010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.542828 (+ 4818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.542829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.545149 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.545152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.547023 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.547024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.547090 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.547094 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.547150 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.547194 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":133,"threads_started":1}
I20260808 10:04:33.550160 9659 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Bootstrap starting.
I20260808 10:04:33.551290 9659 tablet_bootstrap.cc:654] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Neither blocks nor log segments found. Creating new log.
I20260808 10:04:33.551688 9659 log.cc:824] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Log is configured to *not* fsync() on all Append() calls
I20260808 10:04:33.552009 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:42811 (local address 127.0.0.1:59140)
0808 10:04:33.547432 (+ 14803us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.547442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.547480 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.547833 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.547834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.547836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:04:33.547868 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.547869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.549382 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.549384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.550784 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.550787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.551788 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.551789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.551846 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.551849 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.551903 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.551944 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14789,"thread_start_us":131,"threads_started":1}
I20260808 10:04:33.552467 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.24.162.129:42811)
0808 10:04:33.533039 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.533044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.547550 (+ 14506us) server_negotiation.cc:407] Connection header received
0808 10:04:33.547597 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.547600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.547618 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.547699 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:04:33.547950 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.549286 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.550883 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.551619 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.551690 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.551751 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.552348 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.552392 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":55,"threads_started":1}
I20260808 10:04:33.552488 9659 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: No bootstrap required, opened a new log
I20260808 10:04:33.552547 9659 ts_tablet_manager.cc:1397] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:04:33.552836 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.532536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.24.162.131:34043)
0808 10:04:33.537661 (+ 5125us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.537666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.537676 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:04:33.537714 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.537716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.537732 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.537808 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:04:33.538103 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.542694 (+ 4591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.545292 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.546802 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.546904 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.546984 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.552691 (+ 5707us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.552756 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5086,"thread_start_us":73,"threads_started":1}
I20260808 10:04:33.553565 9132 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f (DEFAULT_TABLE table=test-table [id=1ec38f3ef6e045aaa3104df1142e7289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:04:33.553900 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbf8ed2b82648b292baad29796a5f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:04:33.554507 9659 raft_consensus.cc:348] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.554669 9659 raft_consensus.cc:374] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:04:33.554699 9659 raft_consensus.cc:729] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ade997c27742a68cfdfceae4ae9109, State: Initialized, Role: FOLLOWER
I20260808 10:04:33.554806 9659 consensus_queue.cc:260] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.555060 9659 ts_tablet_manager.cc:1428] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:04:33.555181 9429 heartbeater.cc:503] Master 127.24.162.190:33019 was elected leader, sending a full tablet report...
I20260808 10:04:33.557466 9666 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Bootstrap starting.
I20260808 10:04:33.558348 9666 tablet_bootstrap.cc:654] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Neither blocks nor log segments found. Creating new log.
I20260808 10:04:33.558291 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f (DEFAULT_TABLE table=test-table [id=1ec38f3ef6e045aaa3104df1142e7289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:04:33.558643 9666 log.cc:824] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Log is configured to *not* fsync() on all Append() calls
I20260808 10:04:33.558647 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbf8ed2b82648b292baad29796a5f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:04:33.559423 9666 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: No bootstrap required, opened a new log
I20260808 10:04:33.559484 9666 ts_tablet_manager.cc:1397] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:04:33.562114 9666 raft_consensus.cc:348] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.562289 9666 raft_consensus.cc:374] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:04:33.562335 9666 raft_consensus.cc:729] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fadf4d98cf56487183809fdc594b2e53, State: Initialized, Role: FOLLOWER
I20260808 10:04:33.562537 9666 consensus_queue.cc:260] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.562999 9666 ts_tablet_manager.cc:1428] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 10:04:33.563037 9224 heartbeater.cc:503] Master 127.24.162.190:33019 was elected leader, sending a full tablet report...
I20260808 10:04:33.563205 9668 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Bootstrap starting.
I20260808 10:04:33.564162 9668 tablet_bootstrap.cc:654] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Neither blocks nor log segments found. Creating new log.
I20260808 10:04:33.564479 9668 log.cc:824] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Log is configured to *not* fsync() on all Append() calls
I20260808 10:04:33.565276 9668 tablet_bootstrap.cc:492] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: No bootstrap required, opened a new log
I20260808 10:04:33.565357 9668 ts_tablet_manager.cc:1397] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:04:33.567440 9668 raft_consensus.cc:348] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.567605 9668 raft_consensus.cc:374] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:04:33.567646 9668 raft_consensus.cc:729] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1760d6c9d981488f9968042026589be5, State: Initialized, Role: FOLLOWER
I20260808 10:04:33.567775 9668 consensus_queue.cc:260] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.568048 9668 ts_tablet_manager.cc:1428] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:04:33.568168 9630 heartbeater.cc:503] Master 127.24.162.190:33019 was elected leader, sending a full tablet report...
Aug 08 10:04:33 dist-test-slave-nfb5 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
W20260808 10:04:33.687909 9225 tablet.cc:2369] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:04:33.721304 9431 tablet.cc:2369] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:04:33.742544 9631 tablet.cc:2369] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:04:33.786299 9669 raft_consensus.cc:482] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:04:33.786458 9669 raft_consensus.cc:504] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.786870 9669 leader_election.cc:288] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5ade997c27742a68cfdfceae4ae9109 (127.24.162.130:46473), 1760d6c9d981488f9968042026589be5 (127.24.162.131:34043)
I20260808 10:04:33.791507 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.787367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45183 (local address 127.24.162.131:34043)
0808 10:04:33.787412 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.787414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.787633 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:04:33.787750 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.787751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.787760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.787795 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:04:33.788027 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.788793 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.790382 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.790981 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.791032 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.791041 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.791394 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.791436 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:04:33.791507 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.787463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:44039 (local address 127.24.162.130:46473)
0808 10:04:33.787517 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.787519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.787700 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:04:33.787814 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.787815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.787825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.787857 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:04:33.788109 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.788818 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.790353 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.790940 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.790993 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.791003 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.791372 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.791433 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:04:33.791826 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.787142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:34043 (local address 127.24.162.129:45183)
0808 10:04:33.787564 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.787575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.787671 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.787867 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.787868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.787869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:04:33.787955 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.787957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.788918 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.788919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.790263 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.790266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.791101 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.791102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.791295 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.791299 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.791630 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.791673 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":67,"thread_start_us":97,"threads_started":1}
I20260808 10:04:33.792176 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.787173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46473 (local address 127.24.162.129:44039)
0808 10:04:33.787622 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.787634 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:04:33.787732 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:04:33.787927 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:04:33.787928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:04:33.787930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:04:33.788019 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.788021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.788923 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.788924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.790238 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:04:33.790242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.791066 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.791067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:04:33.791158 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.791163 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:04:33.792036 (+ 873us) client_negotiation.cc:240] Negotiation successful
0808 10:04:33.792081 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":57,"thread_start_us":176,"threads_started":1}
I20260808 10:04:33.792188 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbf8ed2b82648b292baad29796a5f7f" candidate_uuid: "fadf4d98cf56487183809fdc594b2e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1760d6c9d981488f9968042026589be5" is_pre_election: true
I20260808 10:04:33.792374 9566 raft_consensus.cc:2454] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fadf4d98cf56487183809fdc594b2e53 in term 0.
I20260808 10:04:33.792616 9063 leader_election.cc:302] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1760d6c9d981488f9968042026589be5, fadf4d98cf56487183809fdc594b2e53; no voters:
I20260808 10:04:33.792760 9669 raft_consensus.cc:2781] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:04:33.792815 9669 raft_consensus.cc:482] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:04:33.792744 9354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbf8ed2b82648b292baad29796a5f7f" candidate_uuid: "fadf4d98cf56487183809fdc594b2e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5ade997c27742a68cfdfceae4ae9109" is_pre_election: true
I20260808 10:04:33.792841 9669 raft_consensus.cc:3037] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:04:33.792896 9354 raft_consensus.cc:2454] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fadf4d98cf56487183809fdc594b2e53 in term 0.
I20260808 10:04:33.794075 9669 raft_consensus.cc:504] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.794215 9669 leader_election.cc:288] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [CANDIDATE]: Term 1 election: Requested vote from peers a5ade997c27742a68cfdfceae4ae9109 (127.24.162.130:46473), 1760d6c9d981488f9968042026589be5 (127.24.162.131:34043)
I20260808 10:04:33.794406 9354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbf8ed2b82648b292baad29796a5f7f" candidate_uuid: "fadf4d98cf56487183809fdc594b2e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5ade997c27742a68cfdfceae4ae9109"
I20260808 10:04:33.794462 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbf8ed2b82648b292baad29796a5f7f" candidate_uuid: "fadf4d98cf56487183809fdc594b2e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1760d6c9d981488f9968042026589be5"
I20260808 10:04:33.794487 9354 raft_consensus.cc:3037] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:04:33.794536 9566 raft_consensus.cc:3037] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:04:33.795509 9566 raft_consensus.cc:2454] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fadf4d98cf56487183809fdc594b2e53 in term 1.
I20260808 10:04:33.795560 9354 raft_consensus.cc:2454] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fadf4d98cf56487183809fdc594b2e53 in term 1.
I20260808 10:04:33.795679 9063 leader_election.cc:302] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1760d6c9d981488f9968042026589be5, fadf4d98cf56487183809fdc594b2e53; no voters:
I20260808 10:04:33.795756 9669 raft_consensus.cc:2781] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:04:33.795902 9669 raft_consensus.cc:686] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [term 1 LEADER]: Becoming Leader. State: Replica: fadf4d98cf56487183809fdc594b2e53, State: Running, Role: LEADER
I20260808 10:04:33.796007 9669 consensus_queue.cc:237] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } }
I20260808 10:04:33.796873 8938 catalog_manager.cc:5495] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 reported cstate change: term changed from 0 to 1, leader changed from <none> to fadf4d98cf56487183809fdc594b2e53 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "fadf4d98cf56487183809fdc594b2e53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fadf4d98cf56487183809fdc594b2e53" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42811 } health_report { overall_health: HEALTHY } } }
I20260808 10:04:33.822705 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.819525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.24.162.129:42811)
0808 10:04:33.819594 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:04:33.819596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:04:33.819606 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:04:33.819646 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:04:33.819647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:04:33.819670 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:04:33.819755 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:04:33.819923 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.820695 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.821751 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:04:33.822052 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:04:33.822143 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:04:33.822310 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:04:33.822438 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:04:33.822542 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:04:33.822585 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 10:04:33.822606 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:04:33.824940 9566 raft_consensus.cc:1261] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 1 FOLLOWER]: Refusing update from remote peer fadf4d98cf56487183809fdc594b2e53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:04:33.825018 9354 raft_consensus.cc:1261] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 1 FOLLOWER]: Refusing update from remote peer fadf4d98cf56487183809fdc594b2e53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:04:33.825313 9678 consensus_queue.cc:1035] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1760d6c9d981488f9968042026589be5" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:04:33.825433 9669 consensus_queue.cc:1035] T 2dbf8ed2b82648b292baad29796a5f7f P fadf4d98cf56487183809fdc594b2e53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5ade997c27742a68cfdfceae4ae9109" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:04:33.833745 9686 mvcc.cc:205] Tried to move back new op lower bound from 7316207508784082944 to 7316207508669116416. Current Snapshot: MvccSnapshot[applied={T|T < 7316207508784082944 or (T in {7316207508784082944})}]
I20260808 10:04:33.834235 9687 mvcc.cc:205] Tried to move back new op lower bound from 7316207508784082944 to 7316207508669116416. Current Snapshot: MvccSnapshot[applied={T|T < 7316207508784082944}]
I20260808 10:04:33.839367 8936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:04:33.839469 8936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:04:33.840370 8936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18ebb2d7e4e3408c87ac63b4397b147a: Sending DeleteTablet for 3 replicas of tablet 2dbf8ed2b82648b292baad29796a5f7f
I20260808 10:04:33.840770 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:04:33 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:36926
I20260808 10:04:33.840890 9325 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:04:33 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:49382
I20260808 10:04:33.841248 9132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbf8ed2b82648b292baad29796a5f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:04:33 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:59140
I20260808 10:04:33.841447 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 9042
I20260808 10:04:33.852218 9700 tablet_replica.cc:330] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: stopping tablet replica
I20260808 10:04:33.852451 9700 raft_consensus.cc:2229] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:04:33.852679 9700 raft_consensus.cc:2258] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:04:33.853963 9700 ts_tablet_manager.cc:1905] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:04:33.856258 9700 ts_tablet_manager.cc:1918] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:04:33.856408 9700 log.cc:1196] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/2dbf8ed2b82648b292baad29796a5f7f
I20260808 10:04:33.856750 9700 ts_tablet_manager.cc:1939] T 2dbf8ed2b82648b292baad29796a5f7f P a5ade997c27742a68cfdfceae4ae9109: Deleting consensus metadata
I20260808 10:04:33.857331 8922 catalog_manager.cc:4841] TS a5ade997c27742a68cfdfceae4ae9109 (127.24.162.130:46473): tablet 2dbf8ed2b82648b292baad29796a5f7f (table test-table [id=1ec38f3ef6e045aaa3104df1142e7289]) successfully deleted
I20260808 10:04:33.858569 9699 tablet_replica.cc:330] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: stopping tablet replica
I20260808 10:04:33.858691 9699 raft_consensus.cc:2229] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:04:33.858819 9699 raft_consensus.cc:2258] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:04:33.859114 8920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:42811 (error 108)
I20260808 10:04:33.859424 9699 ts_tablet_manager.cc:1905] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:04:33.860193 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:04:33.860061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:42811 (local address 127.0.0.1:59146)
0808 10:04:33.860112 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:04:33.860156 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:04:33.860417 8920 catalog_manager.cc:4584] TS fadf4d98cf56487183809fdc594b2e53 (127.24.162.129:42811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2dbf8ed2b82648b292baad29796a5f7f: Network error: Client connection negotiation failed: client connection to 127.24.162.129:42811: connect: Connection refused (error 111)
I20260808 10:04:33.860705 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 9236
I20260808 10:04:33.862754 9699 ts_tablet_manager.cc:1918] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:04:33.862881 9699 log.cc:1196] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/2dbf8ed2b82648b292baad29796a5f7f
I20260808 10:04:33.863545 9699 ts_tablet_manager.cc:1939] T 2dbf8ed2b82648b292baad29796a5f7f P 1760d6c9d981488f9968042026589be5: Deleting consensus metadata
I20260808 10:04:33.864048 8920 catalog_manager.cc:4841] TS 1760d6c9d981488f9968042026589be5 (127.24.162.131:34043): tablet 2dbf8ed2b82648b292baad29796a5f7f (table test-table [id=1ec38f3ef6e045aaa3104df1142e7289]) successfully deleted
I20260808 10:04:33.869055 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 9437
I20260808 10:04:33.882036 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 8894
I20260808 10:04:33.893306 25226 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:04:34.237655 25226 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:04:34.237903 25226 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:04:34.582042 25226 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:04:34.582 UTC [8311] LOG: received smart shutdown request
2026-08-08 10:04:34.585 UTC [8311] LOG: background worker "logical replication launcher" (PID 8323) exited with exit code 1
2026-08-08 10:04:34.585 UTC [8317] LOG: shutting down
2026-08-08 10:04:34.760 UTC [8311] LOG: database system is shut down
2026-08-08T10:04:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46957 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:04:34 dist-test-slave-nfb5 krb5kdc[9715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:04:34 dist-test-slave-nfb5 krb5kdc[9715](info): set up 2 sockets
Aug 08 10:04:34 dist-test-slave-nfb5 krb5kdc[9715](info): commencing operation
krb5kdc: starting...
W20260808 10:04:36.090334 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:04:36 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:04:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangeradmin_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangeradmin_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangerlookup_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangerlookup_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
I20260808 10:04:36.271875 25226 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHQacpx/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/postgres -l logfile start
2026-08-08 10:04:38.778 UTC [9761] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:04:38.779 UTC [9761] LOG: listening on IPv4 address "127.24.162.148", port 56925
2026-08-08 10:04:38.782 UTC [9761] LOG: listening on Unix socket "/tmp/.s.PGSQL.56925"
2026-08-08 10:04:38.818 UTC [9765] LOG: database system was shut down at 2026-08-08 10:04:37 UTC
2026-08-08 10:04:38.823 UTC [9761] LOG: database system is ready to accept connections
I20260808 10:04:39.937595 25226 mini_postgres.cc:96] Postgres bound to 56925
2026-08-08 10:04:39.944 UTC [9775] FATAL: database "slave" does not exist
127.24.162.148:56925 - accepting connections
I20260808 10:04:39.945331 25226 mini_ranger.cc:161] Starting Ranger...
I20260808 10:04:39.961861 25226 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:04:40.436249 25226 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:04:40.436348 25226 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin
2026-08-08 10:04:40,719 [I] DB FLAVOR :POSTGRES
2026-08-08 10:04:40,720 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:04:40,720 [I] Checking connection..
2026-08-08 10:04:40,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:04:40,967 [I] Checking connection passed.
2026-08-08 10:04:40,968 [I] --------- Verifying version history table ---------
2026-08-08 10:04:40,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:04:41,234 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:04:41,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:04:41,480 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:04:41,480 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:04:41,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:04:41.742 UTC [9871] WARNING: there is no transaction in progress
2026-08-08 10:04:41,750 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:04:41,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:04:42,020 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:04:42,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:04:42,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:04:42,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:04:42,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nfb5.c.gcp-upstream.internal','N') ;"
2026-08-08 10:04:42,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:04:42,998 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:04:42,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:04:43,244 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:04:43,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:04:43,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:04:43,742 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:04:43,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:04:44.751 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.768 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.784 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.801 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.817 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.835 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.888 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.895 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.908 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.918 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.928 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.941 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.951 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.957 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.964 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:44.971 UTC [10022] WARNING: there is no transaction in progress
2026-08-08 10:04:45,599 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:04:45,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:04:45,849 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:04:45,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:04:46,087 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:04:46,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:04:46,339 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:04:46,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:04:46,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:04:46,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nfb5.c.gcp-upstream.internal';"
2026-08-08 10:04:47,043 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:04:47,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:04:47,293 [I] DB_PATCHES have already been applied
I20260808 10:04:47.299335 25226 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHQacpx/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:04:47.299412 25226 mini_ranger.cc:193] Using host: 127.24.162.148
I20260808 10:04:47.303313 25226 mini_ranger.cc:227] Ranger admin URL: http://127.24.162.148:41449
Aug 08, 2026 10:04:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:04:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:04:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:04:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:04:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:04:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:04:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:04:48.635923 25226 mini_ranger.cc:160] Time spent starting Ranger: real 8.691s user 0.000s sys 0.010s
Aug 08, 2026 10:05:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:05:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:05:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:05:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:05:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:05:09.442615 25226 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.162.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.162.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangerkms_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.162.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/rangerkms_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.162.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.162.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.162.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/HTTP_127.24.162.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:05:09 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183509, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:05:09.543820 25226 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:05:09.555825 25226 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:05:10.407687 25226 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:05:10.407763 25226 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms
I20260808 10:05:10.407775 25226 mini_ranger_kms.cc:227] Using postgres at 127.24.162.148:56925
2026-08-08 10:05:10,580 [I] DB FLAVOR :POSTGRES
2026-08-08 10:05:10,580 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:05:10,580 [I] Checking connection
2026-08-08 10:05:10,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:05:10,819 [I] connection success
2026-08-08 10:05:10,820 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:05:10,820 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:05:10,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:05:11,078 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:05:11,079 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:05:11,079 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:05:11,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.162.148:56925/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:05:11,341 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:05:11.818118 25226 mini_ranger_kms.cc:310] Created kms service
I20260808 10:05:12.014720 25226 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:05:12.060053 25226 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:05:12 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183509, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:12.461149 25226 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:05:12.461264 25226 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHQacpx/build/release/bin/postgresql.jar:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHQacpx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:05:12.461284 25226 mini_ranger_kms.cc:240] Using host: 127.24.162.148
I20260808 10:05:12.465428 25226 mini_ranger_kms.cc:276] Ranger KMS PID: 10291
I20260808 10:05:12.465513 25226 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.162.148:41471
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:05:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:05:14 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:15.184612 25226 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.641s user 0.006s sys 0.151s
I20260808 10:05:15.184710 25226 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:05:15.184768 25226 mini_ranger_kms.cc:401] 127.24.162.148:41471/kms/v1/keys
Aug 08 10:05:15 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
Aug 08 10:05:15 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183514, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.162.148@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
Aug 08 10:05:17 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:05:17.898018 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35397
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:35397
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:17.985734 10364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:17.985937 10364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:17.985960 10364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:17.987138 10364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:17.987180 10364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:17.987195 10364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:17.987207 10364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:17.987219 10364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:17.988746 10364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.162.148:41471/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:35397
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:17.988938 10364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:17.989104 10364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:17.994056 10376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:17.994129 10372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:17.994087 10364 server_base.cc:1034] running on GCE node
W20260808 10:05:17.994071 10371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:17.994539 10364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:17.994876 10364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:17.996083 10364 hybrid_clock.cc:648] HybridClock initialized: now 1786183517996060 us; error 43 us; skew 500 ppm
Aug 08 10:05:17 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:17.998685 10364 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:17.999136 10364 webserver.cc:460] Webserver started at http://127.24.162.190:33287/ using document root <none> and password file <none>
I20260808 10:05:17.999337 10364 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:17.999374 10364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:17.999464 10364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:05:17 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:18.046522 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "5a064a1974734471b5c7ba652de7cfb0"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "f6562c61a4ffa50d961baf5c1092c16c"
server_key_iv: "e732b5b575658dea7f81e18495c744e6"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.047036 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "5a064a1974734471b5c7ba652de7cfb0"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "f6562c61a4ffa50d961baf5c1092c16c"
server_key_iv: "e732b5b575658dea7f81e18495c744e6"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.048906 10364 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:18.062951 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:18.063189 10364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:18.063284 10364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "5a064a1974734471b5c7ba652de7cfb0"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "f6562c61a4ffa50d961baf5c1092c16c"
server_key_iv: "e732b5b575658dea7f81e18495c744e6"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.063375 10364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:18.074062 10364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:18.076133 10364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:18.076297 10364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:18.081507 10364 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:35397
I20260808 10:05:18.081538 10460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:35397 every 8 connection(s)
I20260808 10:05:18.081954 10364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:18.082715 10461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:18.084520 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 10364
I20260808 10:05:18.084632 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:05:18.084812 25226 external_mini_cluster.cc:1428] Setting key dc7c064b8ed58f27bc3185763ab8eb46
I20260808 10:05:18.085426 10461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Bootstrap starting.
I20260808 10:05:18.086251 10461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:18.086602 10461 log.cc:824] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:18.087482 10461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: No bootstrap required, opened a new log
I20260808 10:05:18.089794 10461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:18.090071 10461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:18.090215 10461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a064a1974734471b5c7ba652de7cfb0, State: Initialized, Role: FOLLOWER
I20260808 10:05:18.090531 10461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:18.090711 10461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:18.090827 10461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:18.090947 10461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:18.091704 10461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:18.091912 10461 leader_election.cc:302] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a064a1974734471b5c7ba652de7cfb0; no voters:
I20260808 10:05:18.092151 10461 leader_election.cc:288] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:05:18.092494 10461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:18.093245 10469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 10:05:18 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:18.093683 10469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 LEADER]: Becoming Leader. State: Replica: 5a064a1974734471b5c7ba652de7cfb0, State: Running, Role: LEADER
I20260808 10:05:18.093933 10469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:18.094722 10461 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:05:18.094782 10473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a064a1974734471b5c7ba652de7cfb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } } }
I20260808 10:05:18.094868 10473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:18.099089 10494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:18.100175 10494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:18.101644 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:18.085383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60174 (local address 127.24.162.190:35397)
0808 10:05:18.085692 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:18.085701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:18.085730 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:05:18.086398 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:18.086410 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:18.086483 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:18.086647 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:18.087441 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.088909 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.090026 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.090654 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.090767 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:18.093890 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:18.093895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:18.093895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:18.093907 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:18.098699 (+ 4792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:18.099237 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:18.099241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:18.099242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:18.099447 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:18.099698 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:18.099699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:18.099700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:18.100174 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:18.100443 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:18.100933 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 10:05:18.101353 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":78,"threads_started":1}
I20260808 10:05:18.099117 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a064a1974734471b5c7ba652de7cfb0. Latest consensus state: current_term: 1 leader_uuid: "5a064a1974734471b5c7ba652de7cfb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } } }
I20260808 10:05:18.101835 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:18.108212 10494 catalog_manager.cc:1347] Generated new cluster ID: 8f64ea8ffa5a4aeaaf765930ec0ad68b
I20260808 10:05:18.108274 10494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:05:18.140921 10494 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:05:18.141743 10494 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:18.148715 10494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Generated new TSK 0
I20260808 10:05:18.149016 10494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:05:18.790282 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:18.885365 10531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:18.885594 10531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:18.885633 10531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:18.886760 10531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:18.886837 10531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:18.886884 10531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:18.888367 10531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:18.888634 10531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:18.888880 10531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:18.894124 10537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:18.894130 10540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:18.894451 10538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:18.895697 10531 server_base.cc:1034] running on GCE node
I20260808 10:05:18.895912 10531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:18.896166 10531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:18.897379 10531 hybrid_clock.cc:648] HybridClock initialized: now 1786183518897345 us; error 53 us; skew 500 ppm
Aug 08 10:05:18 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183518, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:18.900524 10531 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:18.901072 10531 webserver.cc:460] Webserver started at http://127.24.162.129:40387/ using document root <none> and password file <none>
I20260808 10:05:18.901356 10531 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:18.901419 10531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:18.901566 10531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:05:18 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183518, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:18.921839 10531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "13cdb6509395486584364c47510f2b83"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "be7bb04a1a268d65a28b4d9d3e766bb1"
server_key_iv: "551da55fa9c550f2982bd05a9e7eb133"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.922470 10531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "13cdb6509395486584364c47510f2b83"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "be7bb04a1a268d65a28b4d9d3e766bb1"
server_key_iv: "551da55fa9c550f2982bd05a9e7eb133"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.925253 10531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:05:18.942575 10550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:18.943114 10531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 10:05:18.943233 10531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "13cdb6509395486584364c47510f2b83"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "be7bb04a1a268d65a28b4d9d3e766bb1"
server_key_iv: "551da55fa9c550f2982bd05a9e7eb133"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:18.943339 10531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:18.957566 10531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:18.959903 10531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:18.960110 10531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:18.960420 10531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:18.960862 10531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:18.960970 10531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:18.961068 10531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:18.961104 10531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:18.968657 10531 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:46105
I20260808 10:05:18.969017 10716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:46105 every 8 connection(s)
I20260808 10:05:18.969094 10531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:18.972729 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 10531
I20260808 10:05:18.972841 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:05:18.973029 25226 external_mini_cluster.cc:1428] Setting key 94519a60300ca74f88a167b7145c419b
Aug 08 10:05:18 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183518, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:18.990700 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:18.970617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:46001 (local address 127.24.162.190:35397)
0808 10:05:18.971020 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:18.971022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:18.971033 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:05:18.971748 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:18.971749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:18.971765 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:18.971858 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:18.972708 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.973464 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.974468 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.974681 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.974768 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:18.987064 (+ 12296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:18.987066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:18.987066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:18.987075 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:18.989005 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:18.989531 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:18.989532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:18.989532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:18.989602 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:18.989922 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:18.989923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:18.989923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:18.990074 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:18.990118 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:18.990490 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 10:05:18.990584 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":96,"threads_started":1}
I20260808 10:05:18.991561 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:18.970592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.129:46001)
0808 10:05:18.970800 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:18.970849 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:18.971619 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:18.971959 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:18.971963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:18.972212 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:18.972583 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:18.972589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.973558 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.973559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:18.974363 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:18.974365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:18.974805 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:18.974806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:18.974976 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:18.976693 (+ 1717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:18.976703 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:18.983031 (+ 6328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:18.989112 (+ 6081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:18.989120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:18.989123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:18.989435 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:18.989676 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:18.989677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:18.989677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:18.989835 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:18.990164 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:18.990170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:18.990336 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 10:05:18.991137 (+ 801us) client_negotiation.cc:240] Negotiation successful
0808 10:05:18.991316 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260808 10:05:18.992304 10719 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:18.992403 10719 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:18.992597 10719 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:18.993209 10406 ts_manager.cc:194] Registered new tserver with Master: 13cdb6509395486584364c47510f2b83 (127.24.162.129:46105)
I20260808 10:05:18.994148 10406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:46001
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:05:19.021818 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:19.148430 10729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:19.148715 10729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:19.148753 10729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:19.150461 10729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:19.150530 10729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:19.150573 10729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:19.152773 10729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:19.153028 10729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:19.153371 10729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:19.160210 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.160456 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.160602 10744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:19.160808 10729 server_base.cc:1034] running on GCE node
I20260808 10:05:19.161012 10729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:19.161294 10729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:19.162492 10729 hybrid_clock.cc:648] HybridClock initialized: now 1786183519162465 us; error 48 us; skew 500 ppm
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:19.169577 10729 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:19.170275 10729 webserver.cc:460] Webserver started at http://127.24.162.130:35603/ using document root <none> and password file <none>
I20260808 10:05:19.170635 10729 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:19.170778 10729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:19.171060 10729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:19.189369 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "97288949ff614de7aabd9e894c3f1bb1"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "dca7bb09f6fabe4123699d1910069d92"
server_key_iv: "94f0da238e724ea8f4c13ec5abbd1eb5"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.190013 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "97288949ff614de7aabd9e894c3f1bb1"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "dca7bb09f6fabe4123699d1910069d92"
server_key_iv: "94f0da238e724ea8f4c13ec5abbd1eb5"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.194041 10729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:05:19.213798 10753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.214578 10729 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260808 10:05:19.214709 10729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "97288949ff614de7aabd9e894c3f1bb1"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "dca7bb09f6fabe4123699d1910069d92"
server_key_iv: "94f0da238e724ea8f4c13ec5abbd1eb5"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.214824 10729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:19.231426 10729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:19.234390 10729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:19.234597 10729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:19.234839 10729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:19.235244 10729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:19.235291 10729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.235322 10729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:19.235387 10729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.247319 10729 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:38581
I20260808 10:05:19.247884 10729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:19.249230 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 10729
I20260808 10:05:19.249346 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:05:19.249531 25226 external_mini_cluster.cc:1428] Setting key f68d9123dcd0946b0943b7333a2cb7b8
I20260808 10:05:19.249917 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:38581 every 8 connection(s)
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:19.264590 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.250954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:37959 (local address 127.24.162.190:35397)
0808 10:05:19.251002 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:19.251004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:19.251214 (+ 210us) server_negotiation.cc:407] Connection header received
0808 10:05:19.251902 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:19.251903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:19.251919 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:19.251950 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:19.252791 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.253595 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.255385 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.255709 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.255757 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:19.260983 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:19.260984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:19.260985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:19.260993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:19.262885 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:19.263440 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:19.263442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:19.263442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:19.263507 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:19.263808 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:19.263809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:19.263810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:19.263951 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:19.264007 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:19.264404 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 10:05:19.264494 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:05:19.265362 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.250628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.130:37959)
0808 10:05:19.251072 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:19.251119 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:19.251776 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:19.252131 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:19.252135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:19.252403 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:19.252674 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.252680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.253714 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.253714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.255278 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.255281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.255861 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.255862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.255999 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:19.258981 (+ 2982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:19.258992 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:19.260861 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:19.263013 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:19.263020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:19.263023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:19.263338 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:19.263583 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:19.263584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:19.263584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:19.263720 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:19.264044 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:19.264051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:19.264229 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 10:05:19.264947 (+ 718us) client_negotiation.cc:240] Negotiation successful
0808 10:05:19.265116 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":101,"threads_started":1}
I20260808 10:05:19.266053 10914 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:19.266151 10914 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:19.266351 10914 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:19.266700 10406 ts_manager.cc:194] Registered new tserver with Master: 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581)
I20260808 10:05:19.267344 10406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:37959
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:05:19.295902 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:19.410714 10922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:19.410974 10922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:19.411023 10922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:19.412351 10922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:19.412412 10922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:19.412449 10922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:19.414017 10922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:19.414258 10922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:19.414544 10922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:19.419651 10930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.419654 10927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.420038 10928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:19.420861 10922 server_base.cc:1034] running on GCE node
I20260808 10:05:19.421093 10922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:19.421389 10922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:19.422614 10922 hybrid_clock.cc:648] HybridClock initialized: now 1786183519422595 us; error 38 us; skew 500 ppm
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:19.426153 10922 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:19.426771 10922 webserver.cc:460] Webserver started at http://127.24.162.131:34489/ using document root <none> and password file <none>
I20260808 10:05:19.427090 10922 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:19.427203 10922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:19.427372 10922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:19.440924 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "0715f8dc2b43df0e625c2a0bd0d1f07a"
server_key_iv: "deab799bcf428f0405686380e5479811"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.442369 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "0715f8dc2b43df0e625c2a0bd0d1f07a"
server_key_iv: "deab799bcf428f0405686380e5479811"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.452112 10922 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.000s
I20260808 10:05:19.465322 10942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.465628 10922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:19.465742 10922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "0715f8dc2b43df0e625c2a0bd0d1f07a"
server_key_iv: "deab799bcf428f0405686380e5479811"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.465838 10922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:19.484926 10922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:19.494989 10922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:19.497119 10922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:19.497457 10922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:19.497880 10922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:19.497938 10922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.498008 10922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:19.498037 10922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.505240 10922 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:32861
I20260808 10:05:19.505829 10922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:05:19.507362 11106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:32861 every 8 connection(s)
I20260808 10:05:19.513584 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 10922
I20260808 10:05:19.513690 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:05:19.513866 25226 external_mini_cluster.cc:1428] Setting key 2d3fd2f60169f52448760021fafbda50
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:19.521821 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.508228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:45219 (local address 127.24.162.190:35397)
0808 10:05:19.508287 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:19.508289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:19.508564 (+ 275us) server_negotiation.cc:407] Connection header received
0808 10:05:19.509396 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:19.509397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:19.509410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:19.509469 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:19.510095 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.510596 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.511732 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.511926 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.512015 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:19.515745 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:19.515746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:19.515746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:19.515754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:19.517310 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:19.517736 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:19.517738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:19.517738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:19.517802 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:19.521031 (+ 3229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:19.521033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:19.521033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:19.521170 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:19.521253 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:19.521595 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 10:05:19.521708 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:05:19.522436 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.508187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.131:45219)
0808 10:05:19.508384 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:19.508453 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:19.509268 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:19.509546 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:19.509550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:19.509769 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:19.509969 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.509973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.510706 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.510708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.511613 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.511615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.512037 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.512039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.512201 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:19.514011 (+ 1810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:19.514022 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:19.515632 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:19.517410 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:19.517417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:19.517419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:19.517656 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:19.520813 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:19.520814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:19.520814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:19.520948 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:19.521259 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:19.521267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:19.521413 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 10:05:19.522009 (+ 596us) client_negotiation.cc:240] Negotiation successful
0808 10:05:19.522189 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260808 10:05:19.523200 11107 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:19.523306 11107 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:19.523496 11107 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:19.523949 10406 ts_manager.cc:194] Registered new tserver with Master: 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861)
I20260808 10:05:19.524456 10406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:45219
I20260808 10:05:19.525779 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:05:19.526415 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 10531
I20260808 10:05:19.563484 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 10729
I20260808 10:05:19.572031 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 10922
I20260808 10:05:19.583500 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 10364
I20260808 10:05:19.602141 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35397
--webserver_interface=127.24.162.190
--webserver_port=33287
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:35397
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:19.685328 11112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:19.685530 11112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:19.685551 11112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:19.686599 11112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:19.686632 11112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:19.686646 11112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:19.686658 11112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:19.686671 11112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:19.688246 11112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:35397
--ranger_config_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=33287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:19.688424 11112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:19.688585 11112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:19.693360 11119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.693365 11118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.693382 11122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:19.693692 11112 server_base.cc:1034] running on GCE node
I20260808 10:05:19.693938 11112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:19.694222 11112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:19.695389 11112 hybrid_clock.cc:648] HybridClock initialized: now 1786183519695378 us; error 45 us; skew 500 ppm
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:19.700321 11112 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:19.700757 11112 webserver.cc:460] Webserver started at http://127.24.162.190:33287/ using document root <none> and password file <none>
I20260808 10:05:19.700975 11112 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:19.701053 11112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:19.769896 11112 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:19.770788 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.771034 11112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:19.771142 11112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "5a064a1974734471b5c7ba652de7cfb0"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "f6562c61a4ffa50d961baf5c1092c16c"
server_key_iv: "e732b5b575658dea7f81e18495c744e6"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.771504 11112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:19.783460 11112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:19.785650 11112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:19.785776 11112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:19.790302 11112 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:35397
I20260808 10:05:19.790356 11200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:35397 every 8 connection(s)
I20260808 10:05:19.790730 11112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:19.792059 11201 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:05:19.792670 11201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Bootstrap starting.
I20260808 10:05:19.798633 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 11112
I20260808 10:05:19.799053 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:46105
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:35397
--webserver_port=40387
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:19.801014 11201 log.cc:824] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:19.802776 11201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:19.803150 11201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Bootstrap complete.
I20260808 10:05:19.805603 11201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:19.805861 11201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a064a1974734471b5c7ba652de7cfb0, State: Initialized, Role: FOLLOWER
I20260808 10:05:19.806064 11201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:19.806144 11201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:19.806198 11201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:19.806247 11201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:05:19.807338 11201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:19.807464 11201 leader_election.cc:302] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a064a1974734471b5c7ba652de7cfb0; no voters:
I20260808 10:05:19.807670 11201 leader_election.cc:288] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:05:19.807708 11208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:05:19.807834 11208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [term 2 LEADER]: Becoming Leader. State: Replica: 5a064a1974734471b5c7ba652de7cfb0, State: Running, Role: LEADER
I20260808 10:05:19.807951 11201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:19.808015 11208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } }
I20260808 10:05:19.808382 11211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5a064a1974734471b5c7ba652de7cfb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } } }
I20260808 10:05:19.808462 11211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:19.808678 11210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a064a1974734471b5c7ba652de7cfb0. Latest consensus state: current_term: 2 leader_uuid: "5a064a1974734471b5c7ba652de7cfb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a064a1974734471b5c7ba652de7cfb0" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35397 } } }
I20260808 10:05:19.808811 11210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:19.809618 11201 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:05:19.811830 11215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:19.812834 11215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:19.813186 11215 catalog_manager.cc:1259] Loaded cluster ID: 8f64ea8ffa5a4aeaaf765930ec0ad68b
I20260808 10:05:19.813287 11215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:05:19.815217 11215 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:19.815518 11215 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Loaded TSK: 0
I20260808 10:05:19.815809 11215 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:05:19.890553 11204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:19.890969 11204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:19.891058 11204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:19.892808 11204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:19.892907 11204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:19.892997 11204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:19.894765 11204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:46105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=40387
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:19.895040 11204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:19.895301 11204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:19.900780 11254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:19.901192 11204 server_base.cc:1034] running on GCE node
W20260808 10:05:19.900866 11253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:19.901068 11256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:19.902141 11204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:19.902400 11204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:19.909767 11204 hybrid_clock.cc:648] HybridClock initialized: now 1786183519909735 us; error 46 us; skew 500 ppm
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:19.912436 11204 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:19.912952 11204 webserver.cc:460] Webserver started at http://127.24.162.129:40387/ using document root <none> and password file <none>
I20260808 10:05:19.913210 11204 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:19.913298 11204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:19.942579 11204 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260808 10:05:19.943804 11264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.944043 11204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:19.944200 11204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "13cdb6509395486584364c47510f2b83"
format_stamp: "Formatted at 2026-08-08 10:05:18 on dist-test-slave-nfb5"
server_key: "be7bb04a1a268d65a28b4d9d3e766bb1"
server_key_iv: "551da55fa9c550f2982bd05a9e7eb133"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:19.944712 11204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:19.970855 11204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:19.973510 11204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:19.973742 11204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:19.974068 11204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:19.974498 11204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:19.974589 11204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.974704 11204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:19.974787 11204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:19.981745 11204 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:46105
I20260808 10:05:19.982285 11204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:19.984033 11425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:46105 every 8 connection(s)
I20260808 10:05:19.986846 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 11204
I20260808 10:05:19.987167 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:38581
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:35397
--webserver_port=35603
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:05:19 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:20.018337 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.983211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.129:45523)
0808 10:05:19.984964 (+ 1753us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:19.985008 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:19.985626 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:19.991756 (+ 6130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:19.991761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:19.992036 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:19.992371 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.992379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.994611 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.994612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.996040 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:19.996044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.997252 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.997254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:19.997420 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:19.998119 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:19.998130 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:19.999863 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:20.003316 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.003324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.003326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.003612 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.016746 (+ 13134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.016748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.016748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.016966 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.017567 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:20.017577 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:20.017774 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 10:05:20.017903 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 10:05:20.018093 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"thread_start_us":100,"threads_started":1}
I20260808 10:05:20.018576 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:19.983256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45523 (local address 127.24.162.190:35397)
0808 10:05:19.991188 (+ 7932us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:19.991201 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:19.991250 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:05:19.991433 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:19.991441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:19.991506 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:19.992533 (+ 1027us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:19.992627 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.994450 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.996392 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:19.997104 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:19.998946 (+ 1842us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.000008 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:20.000013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:20.000013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:20.000025 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:20.003197 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.005385 (+ 2188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.005388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.005389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.006013 (+ 624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.017082 (+ 11069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.017084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.017084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.017436 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:20.018180 (+ 744us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:20.018261 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:05:20.018387 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7859,"thread_start_us":103,"threads_started":1}
I20260808 10:05:20.021245 11426 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:20.021363 11426 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.021572 11426 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.022514 11426 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:35397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:05:20.022600 11426 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.022670 11426 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:20.023144 11426 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.023221 11426 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.023583 11426 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:05:20.100941 11433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:20.101328 11433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:20.101436 11433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:20.103057 11433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:20.103120 11433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:20.103166 11433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:20.104758 11433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:38581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=35603
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:20.105202 11433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:20.105556 11433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:20.112263 11440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:20.112311 11439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:20.112540 11443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:20.113648 11433 server_base.cc:1034] running on GCE node
I20260808 10:05:20.113883 11433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:20.114141 11433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:20.115305 11433 hybrid_clock.cc:648] HybridClock initialized: now 1786183520115292 us; error 32 us; skew 500 ppm
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:20.117995 11433 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:20.118500 11433 webserver.cc:460] Webserver started at http://127.24.162.130:35603/ using document root <none> and password file <none>
I20260808 10:05:20.118732 11433 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:20.118821 11433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:20.140015 11433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:20.141017 11451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.141209 11433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:20.141330 11433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "97288949ff614de7aabd9e894c3f1bb1"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "dca7bb09f6fabe4123699d1910069d92"
server_key_iv: "94f0da238e724ea8f4c13ec5abbd1eb5"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:20.141743 11433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:20.170241 11433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:20.172577 11433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:20.172755 11433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:20.173008 11433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:20.173377 11433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:20.173434 11433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.173506 11433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:20.173555 11433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.181174 11433 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:38581
I20260808 10:05:20.181241 11596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:38581 every 8 connection(s)
I20260808 10:05:20.181623 11433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:20.183907 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 11433
I20260808 10:05:20.184211 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.162.148:41471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:32861
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:35397
--webserver_port=34489
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:20.193867 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:20.182678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.130:60395)
0808 10:05:20.182858 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:20.182900 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:20.183413 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:20.183664 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:20.183668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:20.183937 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:20.184233 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:20.184241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.185270 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.185271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:20.186501 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:20.186504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.187132 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.187134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:20.187292 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.187939 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:20.187950 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:20.189702 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:20.191965 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.191972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.191975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.192319 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.192556 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.192557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.192557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.192701 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.193056 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:20.193064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:20.193255 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 10:05:20.193408 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 10:05:20.193577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260808 10:05:20.194489 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:20.182660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:60395 (local address 127.24.162.190:35397)
0808 10:05:20.182708 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:20.182711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:20.183021 (+ 310us) server_negotiation.cc:407] Connection header received
0808 10:05:20.183527 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:20.183528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:20.183547 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:20.183598 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:20.184368 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.185126 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.186596 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.186954 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.187023 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.189824 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:20.189826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:20.189827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:20.189835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:20.191861 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.192415 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.192416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.192416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.192480 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.192784 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.192785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.192785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.192954 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:20.193024 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:20.194243 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0808 10:05:20.194350 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:05:20.195534 11597 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:20.195650 11597 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.195859 11597 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.196681 11597 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:35397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:05:20.196776 11597 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.196864 11597 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:20.197321 11597 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.197414 11597 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.197764 11597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:05:20.286818 11603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:20.287151 11603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:20.287204 11603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:20.288357 11603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:20.288427 11603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:20.288473 11603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:20.289940 11603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.162.148:41471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:32861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=34489
--tserver_master_addrs=127.24.162.190:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:20.290205 11603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:20.290484 11603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:20.296219 11612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:20.296310 11608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:20.297348 11603 server_base.cc:1034] running on GCE node
W20260808 10:05:20.298731 11609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:20.298998 11603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:20.299302 11603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:20.300452 11603 hybrid_clock.cc:648] HybridClock initialized: now 1786183520300437 us; error 31 us; skew 500 ppm
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:20.303253 11603 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:20.303660 11603 webserver.cc:460] Webserver started at http://127.24.162.131:34489/ using document root <none> and password file <none>
I20260808 10:05:20.303896 11603 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:20.303978 11603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:20.327685 11603 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 10:05:20.328735 11624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.328958 11603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:20.329041 11603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d"
format_stamp: "Formatted at 2026-08-08 10:05:19 on dist-test-slave-nfb5"
server_key: "0715f8dc2b43df0e625c2a0bd0d1f07a"
server_key_iv: "deab799bcf428f0405686380e5479811"
server_key_version: "kuduclusterkey@0"
I20260808 10:05:20.329511 11603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:20.344870 11603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:20.347498 11603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:20.347710 11603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:20.347994 11603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:20.348428 11603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:20.348491 11603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.348563 11603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:20.348593 11603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:20.355883 11603 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:32861
I20260808 10:05:20.355967 11801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:32861 every 8 connection(s)
I20260808 10:05:20.356403 11603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:05:20.359627 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 11603
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:20.370088 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:20.360037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.24.162.190:35397)
0808 10:05:20.360350 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:20.360352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:20.360358 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:05:20.361510 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:20.361511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:20.361523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:20.361597 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:20.363021 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.363799 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.364930 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.365233 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.365326 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.366807 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:20.366809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:20.366809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:20.366817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:20.368867 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369246 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.369247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.369247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.369315 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369534 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.369535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.369535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.369677 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:20.369840 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:20.369878 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 10:05:20.369966 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":70,"threads_started":1}
I20260808 10:05:20.370772 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:20.357669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:43595 (local address 127.24.162.190:35397)
0808 10:05:20.357716 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:20.357718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:20.359202 (+ 1484us) server_negotiation.cc:407] Connection header received
0808 10:05:20.360260 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:20.360262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:20.360276 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:20.360310 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:20.361200 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.361861 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.363410 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.363621 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.363715 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.367185 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:20.367186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:20.367187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:20.367193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:20.368962 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369567 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.369567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.369568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.369609 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369925 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:20.369926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:20.369927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:20.370059 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:20.370114 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:20.370561 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 10:05:20.370668 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:05:20.370994 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:20.357332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:35397 (local address 127.24.162.131:43595)
0808 10:05:20.359046 (+ 1714us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:20.359096 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:20.359792 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:20.360514 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:20.360518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:20.360784 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:20.361075 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:20.361083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.361949 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.361950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:20.363291 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:20.363294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:20.363751 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:20.363759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:20.363903 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:20.364517 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:20.364528 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:20.367111 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:20.369053 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.369063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.369455 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.369682 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:20.369683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:20.369684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:20.369829 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:20.370150 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:20.370156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:20.370329 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 10:05:20.370468 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 10:05:20.370725 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1668,"thread_start_us":140,"threads_started":1}
I20260808 10:05:20.371603 11802 heartbeater.cc:347] Connected to a master server at 127.24.162.190:35397
I20260808 10:05:20.371714 11802 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.371919 11802 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.372653 11802 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:35397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:05:20.372738 11802 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.372804 11802 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:20.373139 11802 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:20.373240 11802 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
W20260808 10:05:20.373553 11802 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 10:05:20 dist-test-slave-nfb5 krb5kdc[9715](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:21.023783 11426 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:21.023957 11426 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:21.024423 11156 ts_manager.cc:194] Registered new tserver with Master: 13cdb6509395486584364c47510f2b83 (127.24.162.129:46105)
I20260808 10:05:21.025449 11156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:45523
I20260808 10:05:21.197957 11597 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:21.198118 11597 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:21.198426 11156 ts_manager.cc:194] Registered new tserver with Master: 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581)
I20260808 10:05:21.199003 11156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:60395
Aug 08 10:05:21 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
I20260808 10:05:21.373713 11802 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:21.373871 11802 heartbeater.cc:511] Master 127.24.162.190:35397 requested a full tablet report, sending...
I20260808 10:05:21.374184 11156 ts_manager.cc:194] Registered new tserver with Master: 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861)
I20260808 10:05:21.374701 11156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:43595
I20260808 10:05:21.379279 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:05:21.390852 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.383220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.24.162.190:35397)
0808 10:05:21.383641 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.383643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.383756 (+ 113us) server_negotiation.cc:407] Connection header received
0808 10:05:21.383934 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.383935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.383948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.383996 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:21.384976 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.385483 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.386257 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.386472 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.386575 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.387892 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:21.387893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:21.387894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:21.387901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:21.389568 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:21.389967 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:21.389969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:21.389969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:21.390032 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:21.390242 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:21.390243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:21.390243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:21.390359 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:21.390531 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.390630 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.390721 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":99,"threads_started":1}
I20260808 10:05:21.393062 11156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:05:21.394965 11156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:05:21.410166 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.404766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:32861 (local address 127.0.0.1:57750)
0808 10:05:21.405255 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.405299 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.405426 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.405988 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.405992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.405994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.406081 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.406083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.407470 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.407471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.408823 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.408827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.409829 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.409830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.409923 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.409943 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.409992 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.410049 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":93,"threads_started":1}
I20260808 10:05:21.410364 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.405009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:38581 (local address 127.0.0.1:33284)
0808 10:05:21.405760 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.405771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.405814 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.406087 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.406088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.406090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.406123 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.406125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.407791 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.407792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.408853 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.408854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.410142 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.410143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.410202 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.410206 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.410238 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.410281 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":67,"threads_started":1}
I20260808 10:05:21.410796 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.405053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.24.162.131:32861)
0808 10:05:21.405792 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.405797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.405808 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:05:21.405855 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.405858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.405875 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.406162 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.406253 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.407358 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.408940 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.409711 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.410565 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.410632 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.410661 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.410717 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":55,"threads_started":1}
I20260808 10:05:21.411018 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.405143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.24.162.130:38581)
0808 10:05:21.405936 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.405940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.405950 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:05:21.405980 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.405982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.405997 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.406197 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.406282 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.407701 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.408977 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.410055 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.410431 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.410844 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.410874 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.410942 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":62,"threads_started":1}
I20260808 10:05:21.411753 11512 tablet_service.cc:1467] Processing CreateTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c (DEFAULT_TABLE table=test-table [id=bbeb6d1815184d6485e4573de331f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:21.411780 11667 tablet_service.cc:1467] Processing CreateTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c (DEFAULT_TABLE table=test-table [id=bbeb6d1815184d6485e4573de331f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:21.412159 11667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593f9839ed6a45d9ac4c7a34f344c22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:21.412173 11512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593f9839ed6a45d9ac4c7a34f344c22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:21.415489 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.404766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46105 (local address 127.0.0.1:47842)
0808 10:05:21.405523 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.405533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.405633 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.411563 (+ 5930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.411564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.411566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.411611 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.411613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.413160 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.413161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.414213 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.414216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.415234 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.415236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.415306 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.415311 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.415357 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.415394 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260808 10:05:21.416222 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.405206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.24.162.129:46105)
0808 10:05:21.411384 (+ 6178us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.411389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.411401 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:05:21.411436 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.411438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.411455 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.411688 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.411768 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.413057 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.414340 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.415128 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.415682 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.415755 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.415783 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.415944 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6134,"thread_start_us":74,"threads_started":1}
I20260808 10:05:21.416397 11849 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Bootstrap starting.
I20260808 10:05:21.416962 11313 tablet_service.cc:1467] Processing CreateTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c (DEFAULT_TABLE table=test-table [id=bbeb6d1815184d6485e4573de331f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:21.417155 11849 tablet_bootstrap.cc:654] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:21.417330 11313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593f9839ed6a45d9ac4c7a34f344c22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:21.417454 11849 log.cc:824] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:21.421226 11853 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Bootstrap starting.
I20260808 10:05:21.421923 11853 tablet_bootstrap.cc:654] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:21.422338 11853 log.cc:824] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:21.423204 11853 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: No bootstrap required, opened a new log
I20260808 10:05:21.423286 11853 ts_tablet_manager.cc:1397] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:21.425823 11853 raft_consensus.cc:348] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.425978 11853 raft_consensus.cc:374] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:21.426010 11853 raft_consensus.cc:729] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13cdb6509395486584364c47510f2b83, State: Initialized, Role: FOLLOWER
I20260808 10:05:21.426541 11853 consensus_queue.cc:260] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.427016 11853 ts_tablet_manager.cc:1428] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 10:05:21.427080 11848 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Bootstrap starting.
I20260808 10:05:21.427232 11426 heartbeater.cc:503] Master 127.24.162.190:35397 was elected leader, sending a full tablet report...
I20260808 10:05:21.427716 11848 tablet_bootstrap.cc:654] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:21.427938 11849 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: No bootstrap required, opened a new log
I20260808 10:05:21.428011 11849 ts_tablet_manager.cc:1397] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260808 10:05:21.428007 11848 log.cc:824] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:21.430586 11849 raft_consensus.cc:348] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.430752 11849 raft_consensus.cc:374] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:21.430789 11849 raft_consensus.cc:729] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ada6ecafbeb4da2bc49fe783abe4a7d, State: Initialized, Role: FOLLOWER
I20260808 10:05:21.430953 11849 consensus_queue.cc:260] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.431221 11849 ts_tablet_manager.cc:1428] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:05:21.431360 11802 heartbeater.cc:503] Master 127.24.162.190:35397 was elected leader, sending a full tablet report...
I20260808 10:05:21.432389 11848 tablet_bootstrap.cc:492] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: No bootstrap required, opened a new log
I20260808 10:05:21.432474 11848 ts_tablet_manager.cc:1397] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 10:05:21.435422 11848 raft_consensus.cc:348] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.435597 11848 raft_consensus.cc:374] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:21.435645 11848 raft_consensus.cc:729] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97288949ff614de7aabd9e894c3f1bb1, State: Initialized, Role: FOLLOWER
I20260808 10:05:21.435793 11848 consensus_queue.cc:260] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.436065 11848 ts_tablet_manager.cc:1428] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 10:05:21.436187 11597 heartbeater.cc:503] Master 127.24.162.190:35397 was elected leader, sending a full tablet report...
W20260808 10:05:21.485944 11427 tablet.cc:2369] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:05:21 dist-test-slave-nfb5 krb5kdc[9715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for HTTP/127.24.162.148@KRBTEST.COM
W20260808 10:05:21.608016 11803 tablet.cc:2369] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:05:21.683441 11598 tablet.cc:2369] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:05:21.898105 11855 raft_consensus.cc:482] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:21.898284 11855 raft_consensus.cc:504] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.898718 11855 leader_election.cc:288] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861), 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581)
I20260808 10:05:21.899233 11859 raft_consensus.cc:482] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:21.899334 11859 raft_consensus.cc:504] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.899618 11859 leader_election.cc:288] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13cdb6509395486584364c47510f2b83 (127.24.162.129:46105), 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581)
I20260808 10:05:21.900614 11860 raft_consensus.cc:482] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:21.900736 11860 raft_consensus.cc:504] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.901048 11860 leader_election.cc:288] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861), 13cdb6509395486584364c47510f2b83 (127.24.162.129:46105)
I20260808 10:05:21.904070 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.899110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:53275 (local address 127.24.162.131:32861)
0808 10:05:21.899174 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.899176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.899287 (+ 111us) server_negotiation.cc:407] Connection header received
0808 10:05:21.899402 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.899403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.899412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.899443 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.899707 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.900446 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.903118 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.903642 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.903691 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.903700 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.903942 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.903984 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:05:21.905102 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.898938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:32861 (local address 127.24.162.129:53275)
0808 10:05:21.899225 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.899235 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.899312 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.899504 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.899505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.899506 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.899624 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.899626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.900595 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.900596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.903010 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.903014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.903777 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.903778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.903864 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.903868 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.904396 (+ 528us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.904998 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1801,"thread_start_us":101,"threads_started":1}
I20260808 10:05:21.905416 11723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "13cdb6509395486584364c47510f2b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" is_pre_election: true
I20260808 10:05:21.905539 11723 raft_consensus.cc:2454] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13cdb6509395486584364c47510f2b83 in term 0.
I20260808 10:05:21.905742 11268 leader_election.cc:302] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ada6ecafbeb4da2bc49fe783abe4a7d, 13cdb6509395486584364c47510f2b83; no voters:
I20260808 10:05:21.905887 11855 raft_consensus.cc:2781] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:05:21.905931 11855 raft_consensus.cc:482] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:21.905952 11855 raft_consensus.cc:3037] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:21.906929 11855 raft_consensus.cc:504] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.907055 11855 leader_election.cc:288] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [CANDIDATE]: Term 1 election: Requested vote from peers 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861), 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581)
I20260808 10:05:21.907409 11723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "13cdb6509395486584364c47510f2b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d"
I20260808 10:05:21.907490 11723 raft_consensus.cc:3037] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:21.907596 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.900153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:33029 (local address 127.24.162.129:46105)
0808 10:05:21.900191 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.900194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.900864 (+ 670us) server_negotiation.cc:407] Connection header received
0808 10:05:21.900956 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.900957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.900966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.900993 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.901204 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.901899 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.904355 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.904906 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.904953 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.904960 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.907495 (+ 2535us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.907536 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":20}
I20260808 10:05:21.907636 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.902091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:56229 (local address 127.24.162.131:32861)
0808 10:05:21.904173 (+ 2082us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.904175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.904184 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:21.904202 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.904203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.904211 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.904257 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.904531 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.905179 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.906655 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.907211 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.907258 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.907264 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.907526 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.907563 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":2051,"thread_start_us":81,"threads_started":1}
I20260808 10:05:21.907847 11881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.899815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46105 (local address 127.24.162.131:33029)
0808 10:05:21.900802 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.900812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.900888 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.901049 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.901049 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.901051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.901122 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.901124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.902312 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.902312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.903732 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.903736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.905266 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.905267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.905339 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.905343 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.907708 (+ 2365us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.907754 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":103,"threads_started":1}
I20260808 10:05:21.907881 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.901222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:32861 (local address 127.24.162.130:56229)
0808 10:05:21.901735 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.901745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.901796 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.904380 (+ 2584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.904381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.904383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.904449 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.904451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.905445 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.905446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.906546 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.906550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.907363 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.907364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.907435 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.907439 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.907742 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.907783 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":83,"threads_started":1}
I20260808 10:05:21.908114 11728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "97288949ff614de7aabd9e894c3f1bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" is_pre_election: true
I20260808 10:05:21.908164 11362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cdb6509395486584364c47510f2b83" is_pre_election: true
I20260808 10:05:21.908290 11362 raft_consensus.cc:2379] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ada6ecafbeb4da2bc49fe783abe4a7d in current term 1: Already voted for candidate 13cdb6509395486584364c47510f2b83 in this term.
I20260808 10:05:21.908442 11723 raft_consensus.cc:2454] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13cdb6509395486584364c47510f2b83 in term 1.
I20260808 10:05:21.909173 11268 leader_election.cc:302] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ada6ecafbeb4da2bc49fe783abe4a7d, 13cdb6509395486584364c47510f2b83; no voters:
I20260808 10:05:21.909232 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.900002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:38581 (local address 127.24.162.131:33933)
0808 10:05:21.902991 (+ 2989us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.903003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.903096 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.904038 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.904039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.904042 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.904102 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.904104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.906233 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.906234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.907383 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.907386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.908950 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.908951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.909024 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.909028 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.909125 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.909161 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2972,"thread_start_us":70,"threads_started":1}
I20260808 10:05:21.909396 11855 raft_consensus.cc:2781] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:21.909473 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.900255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:33933 (local address 127.24.162.130:38581)
0808 10:05:21.903882 (+ 3627us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.903884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.903893 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:21.903912 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.903913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.903929 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.903996 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.905413 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.906129 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.908121 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.908838 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.908909 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.908916 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.909366 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.909401 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3606,"thread_start_us":75,"threads_started":1}
I20260808 10:05:21.909520 11855 raft_consensus.cc:686] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 LEADER]: Becoming Leader. State: Replica: 13cdb6509395486584364c47510f2b83, State: Running, Role: LEADER
I20260808 10:05:21.909600 11855 consensus_queue.cc:237] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } }
I20260808 10:05:21.909793 11515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97288949ff614de7aabd9e894c3f1bb1" is_pre_election: true
I20260808 10:05:21.909883 11515 raft_consensus.cc:2454] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ada6ecafbeb4da2bc49fe783abe4a7d in term 0.
I20260808 10:05:21.910044 11626 leader_election.cc:302] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ada6ecafbeb4da2bc49fe783abe4a7d, 97288949ff614de7aabd9e894c3f1bb1; no voters: 13cdb6509395486584364c47510f2b83
I20260808 10:05:21.910199 11859 raft_consensus.cc:2741] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 10:05:21.910234 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.901423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46105 (local address 127.24.162.130:44729)
0808 10:05:21.901870 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.901879 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.901916 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.902137 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.902138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.902140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.902211 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.902213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.903108 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.903109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.908639 (+ 5530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.908642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.909979 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.909980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.910049 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.910053 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.910102 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.910137 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":65,"threads_started":1}
I20260808 10:05:21.910516 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.901589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:44729 (local address 127.24.162.129:46105)
0808 10:05:21.902006 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.902008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.902014 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:05:21.902031 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.902032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.902040 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.902096 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.902282 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.903022 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.908741 (+ 5719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.909853 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.909931 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.909940 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.910406 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.910445 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":61,"threads_started":1}
I20260808 10:05:21.910477 11155 catalog_manager.cc:5495] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13cdb6509395486584364c47510f2b83 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "13cdb6509395486584364c47510f2b83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13cdb6509395486584364c47510f2b83" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 46105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 } health_report { overall_health: UNKNOWN } } }
I20260808 10:05:21.910717 11362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "97288949ff614de7aabd9e894c3f1bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13cdb6509395486584364c47510f2b83" is_pre_election: true
I20260808 10:05:21.911003 11453 leader_election.cc:302] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97288949ff614de7aabd9e894c3f1bb1; no voters: 0ada6ecafbeb4da2bc49fe783abe4a7d, 13cdb6509395486584364c47510f2b83
I20260808 10:05:21.911167 11860 raft_consensus.cc:2726] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 10:05:21.911286 11879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.898938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:38581 (local address 127.24.162.129:45595)
0808 10:05:21.899134 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.899148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:21.899242 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:21.899472 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:21.899473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:21.899475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:21.899589 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.899593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.900553 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.900554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.901869 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:21.901872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.905127 (+ 3255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.905128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:21.911105 (+ 5977us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.911110 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:05:21.911169 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:05:21.911207 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260808 10:05:21.911337 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.899114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45595 (local address 127.24.162.130:38581)
0808 10:05:21.899161 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.899164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.899210 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:05:21.899345 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.899346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.899357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.899392 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:21.899682 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.900444 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.904479 (+ 4035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.905018 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.905063 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.905069 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.911228 (+ 6159us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.911278 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:05:21.911650 11515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "13cdb6509395486584364c47510f2b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97288949ff614de7aabd9e894c3f1bb1" is_pre_election: true
I20260808 10:05:21.911725 11515 raft_consensus.cc:2454] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13cdb6509395486584364c47510f2b83 in term 0.
I20260808 10:05:21.911988 11518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593f9839ed6a45d9ac4c7a34f344c22c" candidate_uuid: "13cdb6509395486584364c47510f2b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97288949ff614de7aabd9e894c3f1bb1"
I20260808 10:05:21.912073 11518 raft_consensus.cc:3037] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:21.912894 11518 raft_consensus.cc:2454] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13cdb6509395486584364c47510f2b83 in term 1.
I20260808 10:05:21.960872 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.957950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.24.162.129:46105)
0808 10:05:21.958019 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:21.958021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:21.958168 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:05:21.958221 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:21.958222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:21.958245 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:21.958324 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:05:21.958426 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.959213 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.959973 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:21.960280 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:21.960354 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:21.960510 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:05:21.960653 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:05:21.960750 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:21.960767 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 10:05:21.960789 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:05:21.962644 11518 raft_consensus.cc:1261] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 1 FOLLOWER]: Refusing update from remote peer 13cdb6509395486584364c47510f2b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:21.962669 11723 raft_consensus.cc:1261] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 1 FOLLOWER]: Refusing update from remote peer 13cdb6509395486584364c47510f2b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:21.963020 11891 consensus_queue.cc:1035] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ada6ecafbeb4da2bc49fe783abe4a7d" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 32861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:21.963160 11855 consensus_queue.cc:1035] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97288949ff614de7aabd9e894c3f1bb1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 38581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:21.965776 11901 mvcc.cc:205] Tried to move back new op lower bound from 7316207705956384768 to 7316207705742131200. Current Snapshot: MvccSnapshot[applied={T|T < 7316207705956384768}]
I20260808 10:05:21.967959 11902 mvcc.cc:205] Tried to move back new op lower bound from 7316207705956384768 to 7316207705742131200. Current Snapshot: MvccSnapshot[applied={T|T < 7316207705956384768 or (T in {7316207705956384768})}]
I20260808 10:05:21.968933 11156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:21.969034 11156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:21.969736 11156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a064a1974734471b5c7ba652de7cfb0: Sending DeleteTablet for 3 replicas of tablet 593f9839ed6a45d9ac4c7a34f344c22c
I20260808 10:05:21.970036 11667 tablet_service.cc:1514] Processing DeleteTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:21 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:57750
I20260808 10:05:21.970175 11313 tablet_service.cc:1514] Processing DeleteTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:21 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:47842
I20260808 10:05:21.970382 11512 tablet_service.cc:1514] Processing DeleteTablet for tablet 593f9839ed6a45d9ac4c7a34f344c22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:21 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:33284
I20260808 10:05:21.970427 11913 tablet_replica.cc:330] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: stopping tablet replica
I20260808 10:05:21.970523 11913 raft_consensus.cc:2229] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:05:21.970659 11913 raft_consensus.cc:2258] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:21.971009 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 11204
I20260808 10:05:21.971141 11913 ts_tablet_manager.cc:1905] T 593f9839ed6a45d9ac4c7a34f344c22c P 13cdb6509395486584364c47510f2b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:21.979492 11915 tablet_replica.cc:330] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: stopping tablet replica
I20260808 10:05:21.979638 11915 raft_consensus.cc:2229] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:21.979775 11915 raft_consensus.cc:2258] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:21.979971 11912 tablet_replica.cc:330] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: stopping tablet replica
I20260808 10:05:21.980098 11912 raft_consensus.cc:2229] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:21.980214 11912 raft_consensus.cc:2258] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:21.980566 11915 ts_tablet_manager.cc:1905] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:21.982642 11915 ts_tablet_manager.cc:1918] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:21.982731 11915 log.cc:1196] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/593f9839ed6a45d9ac4c7a34f344c22c
I20260808 10:05:21.983038 11915 ts_tablet_manager.cc:1939] T 593f9839ed6a45d9ac4c7a34f344c22c P 97288949ff614de7aabd9e894c3f1bb1: Deleting consensus metadata
I20260808 10:05:21.983536 11137 catalog_manager.cc:4841] TS 97288949ff614de7aabd9e894c3f1bb1 (127.24.162.130:38581): tablet 593f9839ed6a45d9ac4c7a34f344c22c (table test-table [id=bbeb6d1815184d6485e4573de331f825]) successfully deleted
I20260808 10:05:21.983629 11912 ts_tablet_manager.cc:1905] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:05:21.984879 11137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:46105 (error 108)
I20260808 10:05:21.985282 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 11433
I20260808 10:05:21.986119 11912 ts_tablet_manager.cc:1918] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:21.986217 11912 log.cc:1196] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/593f9839ed6a45d9ac4c7a34f344c22c
I20260808 10:05:21.986577 11912 ts_tablet_manager.cc:1939] T 593f9839ed6a45d9ac4c7a34f344c22c P 0ada6ecafbeb4da2bc49fe783abe4a7d: Deleting consensus metadata
I20260808 10:05:21.987110 11138 catalog_manager.cc:4841] TS 0ada6ecafbeb4da2bc49fe783abe4a7d (127.24.162.131:32861): tablet 593f9839ed6a45d9ac4c7a34f344c22c (table test-table [id=bbeb6d1815184d6485e4573de331f825]) successfully deleted
I20260808 10:05:21.989619 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:21.985321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:46105 (local address 127.0.0.1:47874)
0808 10:05:21.989497 (+ 4176us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:21.989565 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:46105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4155,"thread_start_us":39,"threads_started":1}
W20260808 10:05:21.989740 11137 catalog_manager.cc:4584] TS 13cdb6509395486584364c47510f2b83 (127.24.162.129:46105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 593f9839ed6a45d9ac4c7a34f344c22c: Network error: Client connection negotiation failed: client connection to 127.24.162.129:46105: connect: Connection refused (error 111)
I20260808 10:05:21.995234 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 11603
I20260808 10:05:22.005472 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 11112
I20260808 10:05:22.015317 25226 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:05:22.360061 25226 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:05:22.360149 25226 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:05:22.706120 25226 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:05:22.706 UTC [9761] LOG: received smart shutdown request
2026-08-08 10:05:22.711 UTC [9761] LOG: background worker "logical replication launcher" (PID 9771) exited with exit code 1
2026-08-08 10:05:22.711 UTC [9766] LOG: shutting down
2026-08-08 10:05:22.866 UTC [9761] LOG: database system is shut down
2026-08-08T10:05:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48106 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:05:23 dist-test-slave-nfb5 krb5kdc[11932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:05:23 dist-test-slave-nfb5 krb5kdc[11932](info): set up 2 sockets
Aug 08 10:05:23 dist-test-slave-nfb5 krb5kdc[11932](info): commencing operation
krb5kdc: starting...
W20260808 10:05:24.194593 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:05:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:05:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:05:24.322453 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36125
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:24.405575 11951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:24.405823 11951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:24.405864 11951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:24.407085 11951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:24.407147 11951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:24.407172 11951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:24.407203 11951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:24.407233 11951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:24.408716 11951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:24.409013 11951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:24.409250 11951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:24.414300 11963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:24.414310 11966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.414346 11951 server_base.cc:1034] running on GCE node
W20260808 10:05:24.414346 11961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.414819 11951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:24.415201 11951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:24.416401 11951 hybrid_clock.cc:648] HybridClock initialized: now 1786183524416379 us; error 34 us; skew 500 ppm
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:24.419071 11951 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:24.419525 11951 webserver.cc:460] Webserver started at http://127.24.162.190:43681/ using document root <none> and password file <none>
I20260808 10:05:24.419763 11951 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:24.419835 11951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:24.419998 11951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:24.421099 11951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "a99ac375508c4a96b0261d84c2b004c8"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "13532bf7a112caf5584617c1228b680a"
server_key_iv: "7bac1d2e8a02a484f9ebc10a3da5fb19"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.421489 11951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "a99ac375508c4a96b0261d84c2b004c8"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "13532bf7a112caf5584617c1228b680a"
server_key_iv: "7bac1d2e8a02a484f9ebc10a3da5fb19"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.422986 11951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:05:24.423767 11974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:24.423941 11951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:05:24.424046 11951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "a99ac375508c4a96b0261d84c2b004c8"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "13532bf7a112caf5584617c1228b680a"
server_key_iv: "7bac1d2e8a02a484f9ebc10a3da5fb19"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.424132 11951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:24.433677 11951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:24.435604 11951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:24.435775 11951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:24.439961 11951 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36125
I20260808 10:05:24.439996 12050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36125 every 8 connection(s)
I20260808 10:05:24.440354 11951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:24.441241 12052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:24.442814 12052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.447928 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 11951
I20260808 10:05:24.448037 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:05:24.448247 25226 external_mini_cluster.cc:1428] Setting key 397901dd8b38e0df726c3deb08a14220
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:24.452544 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.443718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.24.162.190:36125)
0808 10:05:24.443837 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.443841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.443878 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:05:24.444333 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.444334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.444351 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.444425 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.444939 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.445720 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.446550 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.446860 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.446997 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.449192 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.449200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.449201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.449209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.450826 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.451314 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.451316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.451316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.451395 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.451665 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.451666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.451666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.451879 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.451969 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.452165 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.452298 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":60,"threads_started":1}
I20260808 10:05:24.452551 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.443575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36125 (local address 127.0.0.1:54974)
0808 10:05:24.443726 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.443761 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.444198 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.444458 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.444461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.444705 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.444858 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.444861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.445807 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.445807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.446477 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.446479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.446991 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.446992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.447065 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.447523 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.447531 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.449098 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.450936 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.450944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.450946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.451230 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.451461 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.451462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.451462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.451581 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.451961 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.451966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.452035 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.452130 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.452370 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":98,"thread_start_us":63,"threads_started":1}
I20260808 10:05:24.453352 12052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.453696 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.453584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42518)
0808 10:05:24.453632 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.453675 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:05:24.453840 11979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
I20260808 10:05:24.454253 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.454152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42534)
0808 10:05:24.454198 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.454223 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:05:24.454358 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:43363: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.162.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
I20260808 10:05:24.475234 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:43363
--webserver_interface=127.24.162.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:24.503576 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } attempt: 1
I20260808 10:05:24.504226 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.504052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42542)
0808 10:05:24.504145 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.504192 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 10:05:24.504676 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.504579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42552)
0808 10:05:24.504625 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.504653 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:05:24.504796 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:43363: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
W20260808 10:05:24.559720 12064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:24.560011 12064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:24.560041 12064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:24.561223 12064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:24.561268 12064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:24.561283 12064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:24.561296 12064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:24.561308 12064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:24.562815 12064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:43363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.162.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:24.563102 12064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:24.563351 12064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:24.568382 12070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:24.568311 12074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.568517 12064 server_base.cc:1034] running on GCE node
W20260808 10:05:24.568336 12069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.568926 12064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:24.569231 12064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:24.570400 12064 hybrid_clock.cc:648] HybridClock initialized: now 1786183524570388 us; error 45 us; skew 500 ppm
I20260808 10:05:24.572937 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } attempt: 2
I20260808 10:05:24.573355 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.573228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42566)
0808 10:05:24.573287 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.573326 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:05:24.573688 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.573615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42574)
0808 10:05:24.573651 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.573674 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 10:05:24.573799 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:43363: Network error: Client connection negotiation failed: client connection to 127.24.162.189:43363: connect: Connection refused (error 111)
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:24.576409 12064 init.cc:377] Logged in from keytab as kudu/127.24.162.189@KRBTEST.COM (short username kudu)
I20260808 10:05:24.576798 12064 webserver.cc:460] Webserver started at http://127.24.162.189:42073/ using document root <none> and password file <none>
I20260808 10:05:24.576983 12064 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:24.577023 12064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:24.577114 12064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:24.578150 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data/instance:
uuid: "fe8eefedb3f64c419252e926c674c881"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "5db07e765265863a25502fc007d4fd45"
server_key_iv: "63489a03108679fe9e91d2f950baceaf"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.578464 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal/instance:
uuid: "fe8eefedb3f64c419252e926c674c881"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "5db07e765265863a25502fc007d4fd45"
server_key_iv: "63489a03108679fe9e91d2f950baceaf"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.579820 12064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:05:24.580523 12085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:24.580695 12064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:05:24.580799 12064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "fe8eefedb3f64c419252e926c674c881"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "5db07e765265863a25502fc007d4fd45"
server_key_iv: "63489a03108679fe9e91d2f950baceaf"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.580854 12064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:24.605374 12064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:24.607430 12064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:24.607597 12064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:24.611811 12064 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.189:43363
I20260808 10:05:24.611896 12151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.189:43363 every 8 connection(s)
I20260808 10:05:24.612262 12064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
I20260808 10:05:24.613178 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:24.614737 12152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:24.620829 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12064
I20260808 10:05:24.620941 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-1/wal/instance
I20260808 10:05:24.621119 25226 external_mini_cluster.cc:1428] Setting key 779a545c784fac100f7a05ea2dfed76f
I20260808 10:05:24.623924 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.615490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.24.162.190:36125)
0808 10:05:24.615550 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.615552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.615775 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:05:24.616209 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.616210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.616222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.616295 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.616912 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.617411 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.618342 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.618534 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.618587 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.620768 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.620769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.620769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.620779 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.622474 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.622865 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.622866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.622867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.622956 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.623284 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.623285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.623285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.623391 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.623444 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.623790 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.623857 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:05:24.624205 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.615456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36125 (local address 127.0.0.1:54984)
0808 10:05:24.615641 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.615683 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.616130 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.616381 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.616384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.616631 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.616820 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.616824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.617493 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.617494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.618262 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.618264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.618607 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.618607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.618707 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.619258 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.619267 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.620660 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.622569 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.622575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.622577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.622792 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.623043 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.623044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.623045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.623209 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.623462 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.623468 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.623637 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.623758 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.623941 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260808 10:05:24.624802 12152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.162.188@KRBTEST.COM; defaulting to no policy
I20260808 10:05:24.632328 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.625057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42580)
0808 10:05:24.625108 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.625118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.625153 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.625555 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.625556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.625696 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.625742 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.625744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.626601 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.626602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.627241 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.627243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.627792 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.627793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.627857 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.628298 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.628303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.629494 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.631284 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.631284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.631285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.631454 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.631664 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.631665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.631665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.631721 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.632008 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.632009 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.632047 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.632108 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.632205 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:05:24.632416 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.625136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.24.162.189:43363)
0808 10:05:24.625303 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.625307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.625329 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:05:24.625346 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.625347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.625357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.625553 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.625832 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.626512 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.627329 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.627702 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.627793 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.629578 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.629587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.629587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.629594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.631177 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.631533 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.631536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.631537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.631594 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.631790 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.631791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.631791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.631933 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.632031 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.632146 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.632279 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 10:05:24.633004 12152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.188@KRBTEST.COM" created.
I20260808 10:05:24.633379 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.633250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36428)
0808 10:05:24.633299 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.633351 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:05:24.633514 12090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
I20260808 10:05:24.633926 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.633823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36440)
0808 10:05:24.633872 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.633896 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:05:24.634068 12152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
I20260808 10:05:24.649107 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:38227
--webserver_interface=127.24.162.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:24.664961 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } attempt: 3
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:05:24.672196 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.665423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.24.162.189:43363)
0808 10:05:24.665494 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.665497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.665509 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:05:24.665561 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.665562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.665579 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.665634 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.666025 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.666547 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.667278 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.667493 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.667574 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.669619 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.669621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.669621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.669629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.671019 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.671374 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.671376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.671376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.671437 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.671650 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.671651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.671651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.671766 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.671847 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.672008 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.672090 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:05:24.672196 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.665343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42592)
0808 10:05:24.665408 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.665429 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.665498 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.665656 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.665657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.665832 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.665901 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.665903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.666634 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.666635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.667192 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.667194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.667573 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.667574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.667659 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.668168 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.668173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.669530 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.671097 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.671098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.671099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.671279 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.671506 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.671507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.671507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.671575 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.671841 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.671842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.671893 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.671982 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.672090 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 10:05:24.672636 12052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.673022 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.672923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36452)
0808 10:05:24.672969 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.673000 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:05:24.673323 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.673234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36468)
0808 10:05:24.673275 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.673299 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 10:05:24.673417 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
I20260808 10:05:24.683290 12152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 1
I20260808 10:05:24.683669 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.683556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36482)
0808 10:05:24.683608 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.683643 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:05:24.684001 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.683900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36498)
0808 10:05:24.683949 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.683981 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:05:24.684087 12152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
I20260808 10:05:24.704632 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 1
I20260808 10:05:24.705116 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.704990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36508)
0808 10:05:24.705049 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.705088 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:05:24.705473 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.705375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36510)
0808 10:05:24.705419 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.705447 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:05:24.705561 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
W20260808 10:05:24.736814 12170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:24.737157 12170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:24.737216 12170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:24.738804 12170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:24.738871 12170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:24.738893 12170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:24.738931 12170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:24.738955 12170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:24.741470 12170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36125,127.24.162.189:43363,127.24.162.188:38227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:38227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.162.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:24.742754 12170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:24.743036 12170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:24.748085 12179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:24.748044 12180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.748507 12170 server_base.cc:1034] running on GCE node
W20260808 10:05:24.748044 12185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:24.749068 12170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:24.749460 12170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:24.750655 12170 hybrid_clock.cc:648] HybridClock initialized: now 1786183524750629 us; error 46 us; skew 500 ppm
I20260808 10:05:24.752207 12152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 2
I20260808 10:05:24.752655 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.752519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36520)
0808 10:05:24.752565 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.752613 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 10:05:24.753024 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.752915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36526)
0808 10:05:24.752962 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.752993 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 10:05:24.753121 12152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:24.754540 12170 init.cc:377] Logged in from keytab as kudu/127.24.162.188@KRBTEST.COM (short username kudu)
I20260808 10:05:24.756407 12170 webserver.cc:460] Webserver started at http://127.24.162.188:39247/ using document root <none> and password file <none>
I20260808 10:05:24.756630 12170 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:24.756677 12170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:24.756824 12170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:24.758374 12170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data/instance:
uuid: "bba4af5941094da6965f3994099a3f83"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "35e7c7bc61caf7f7395bfb9d13c4115a"
server_key_iv: "5cf2ec8d172270f3ac162beccee17503"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.758813 12170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal/instance:
uuid: "bba4af5941094da6965f3994099a3f83"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "35e7c7bc61caf7f7395bfb9d13c4115a"
server_key_iv: "5cf2ec8d172270f3ac162beccee17503"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.760927 12170 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:05:24.762012 12195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:24.762197 12170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:24.762291 12170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "bba4af5941094da6965f3994099a3f83"
format_stamp: "Formatted at 2026-08-08 10:05:24 on dist-test-slave-nfb5"
server_key: "35e7c7bc61caf7f7395bfb9d13c4115a"
server_key_iv: "5cf2ec8d172270f3ac162beccee17503"
server_key_version: "encryptionkey@0"
I20260808 10:05:24.762364 12170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:24.780721 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 2
I20260808 10:05:24.781189 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.781066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36542)
0808 10:05:24.781115 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.781158 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 10:05:24.781629 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.781519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36556)
0808 10:05:24.781568 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.781596 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:05:24.781817 12052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38227: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38227: connect: Connection refused (error 111)
I20260808 10:05:24.801862 12170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:24.804159 12170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:24.804327 12170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:24.808386 12170 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.188:38227
I20260808 10:05:24.808454 12276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.188:38227 every 8 connection(s)
I20260808 10:05:24.808822 12170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
I20260808 10:05:24.809755 12278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:24.811507 12278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.814682 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12170
I20260808 10:05:24.814782 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183366714887-25226-0/minicluster-data/master-2/wal/instance
I20260808 10:05:24.814997 25226 external_mini_cluster.cc:1428] Setting key 1fcded964be0dddd1371d1b739ee3b70
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:24.821460 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.812189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36125 (local address 127.0.0.1:54990)
0808 10:05:24.812371 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.812412 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.812855 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.813101 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.813105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.813364 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.813556 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.813559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.814172 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.814173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.815308 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.815310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.815824 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.815824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.815945 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.816411 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.816423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.818246 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.819898 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.819907 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.819910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.820233 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.820437 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.820438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.820438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.820593 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.820944 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.820952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.821091 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.821178 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.821292 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260808 10:05:24.821462 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.812243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.24.162.190:36125)
0808 10:05:24.812306 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.812308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.812499 (+ 191us) server_negotiation.cc:407] Connection header received
0808 10:05:24.812953 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.812954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.812965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.813026 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.813632 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.814095 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.815419 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.815747 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.815818 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.818338 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.818339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.818339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.818346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.819836 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.820310 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.820311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.820312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.820364 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.820731 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.820732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.820733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.820854 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.820923 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.821267 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.821360 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:05:24.822155 12278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.825191 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.815402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36558 (local address 127.24.162.188:38227)
0808 10:05:24.815593 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.815599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.815623 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:05:24.815691 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.815694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.815730 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.815806 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.816871 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.817883 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.818792 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.819325 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.819394 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.822141 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.822147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.822148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.822153 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.823991 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.824371 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.824373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.824373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.824424 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.824637 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.824638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.824638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.824799 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.824946 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.824994 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.825076 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":131,"threads_started":1}
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:05:24.829770 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.822413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:43363 (local address 127.0.0.1:42608)
0808 10:05:24.822475 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.822485 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.822519 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.822672 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.822673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.822846 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.822893 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.822894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.823857 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.823858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.824582 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.824585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.825028 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.825029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.825108 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.825539 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.825543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.826905 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.828755 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.828756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.828756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.828926 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.829149 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.829150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.829150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.829209 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.829470 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.829471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.829512 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.829589 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.829682 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:05:24.829787 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.822495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42608 (local address 127.24.162.189:43363)
0808 10:05:24.822536 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.822538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.822546 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:05:24.822571 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.822571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.822583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.822659 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.823027 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.823757 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.824661 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.824950 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.825019 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.827012 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.827013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.827013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.827020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.828675 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.829012 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.829013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.829013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.829076 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.829279 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.829280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.829280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.829395 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.829465 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.829629 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.829701 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:24.830269 12278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:05:24.833330 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.826057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.24.162.190:36125)
0808 10:05:24.826121 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.826123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.826135 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:05:24.826197 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.826198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.826209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.826273 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.826737 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.827399 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.828156 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.828358 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.828413 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.830713 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.830714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.830714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.830721 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.832163 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.832503 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.832504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.832504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.832560 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.832759 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.832760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.832760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.832866 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.833051 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.833088 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.833162 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:05:24.837033 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.830508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36572)
0808 10:05:24.830561 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.830573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.830622 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.830733 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.830734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.830868 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.830914 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.830916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.831857 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.831858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.832292 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.832294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.832694 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.832695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.832742 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.833152 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.833157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.834503 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.836018 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.836019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.836020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.836181 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.836402 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.836402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.836403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.836463 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.836708 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.836710 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.836747 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.836832 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.836925 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:05:24.837086 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.830544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36572 (local address 127.24.162.188:38227)
0808 10:05:24.830576 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.830578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.830608 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:05:24.830675 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.830676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.830685 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.830733 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.830999 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.831773 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.832412 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.832621 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.832696 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.834582 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.834583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.834583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.834589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.835922 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.836269 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.836270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.836270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.836327 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.836534 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.836535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.836535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.836636 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.836705 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.836857 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.836975 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":16}
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:05:24.838649 12278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: Bootstrap starting.
I20260808 10:05:24.839603 12278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:24.839910 12278 log.cc:824] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:24.840794 12278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: No bootstrap required, opened a new log
I20260808 10:05:24.842025 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.834220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.24.162.189:43363)
0808 10:05:24.834258 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.834260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.834266 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:05:24.834282 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.834282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.834289 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.834328 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.834888 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.835584 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.836244 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.836446 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.836518 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.839110 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.839112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.839113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.839119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.840914 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.841241 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.841242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.841242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.841303 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.841502 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.841502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.841503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.841636 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.841815 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.841857 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.841932 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:05:24.842875 12278 raft_consensus.cc:348] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.843076 12278 raft_consensus.cc:374] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:24.843129 12278 raft_consensus.cc:729] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba4af5941094da6965f3994099a3f83, State: Initialized, Role: FOLLOWER
I20260808 10:05:24.843248 12278 consensus_queue.cc:260] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.843593 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:05:24.843611 12278 sys_catalog.cc:564] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:24.843559 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.843724 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:05:24.844309 12152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 3
W20260808 10:05:24.845527 12309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:05:24.845651 12309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:05:24.845723 12170 master_runner.cc:186] Error getting master registration for 127.24.162.190:36125: OK, instance_id { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" instance_seqno: 1786183524438703 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:05:24.848673 12170 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:05:24.852864 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.844535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36576)
0808 10:05:24.844596 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.844608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.844660 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.845169 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.845170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.845351 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.845397 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.845399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.846335 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.846336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.847145 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.847149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.847728 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.847729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.847784 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.848195 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.848201 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.849809 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.851751 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.851752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.851753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.851969 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.852180 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.852181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.852181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.852241 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.852557 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.852558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.852601 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.852667 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.852770 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:05:24.853631 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.845092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.24.162.190:36125)
0808 10:05:24.845132 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.845134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.845722 (+ 588us) server_negotiation.cc:407] Connection header received
0808 10:05:24.845811 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.845812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.845821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.845848 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.846709 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.847470 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.848749 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.849039 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.849092 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.850673 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.850674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.850674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.850681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.852439 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.852834 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.852835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.852835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.852891 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.853113 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.853115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.853115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.853229 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.853331 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.853437 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.853519 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:05:24.853662 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.844989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.24.162.188:38227)
0808 10:05:24.845031 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.845033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.845042 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:24.845064 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.845065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.845075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.845473 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.845504 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.846232 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.847260 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.847610 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.847678 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.849897 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.849898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.849899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.849905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.851653 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.852052 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.852053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.852054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.852111 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.852304 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.852305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.852305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.852464 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.852528 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.853504 (+ 976us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.853583 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:05:24.855896 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.846484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.24.162.189:43363)
0808 10:05:24.846527 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.846529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.846786 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:05:24.846871 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.846871 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.846880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.846909 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.847640 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.848314 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.849257 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.849556 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.849606 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.852215 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.852216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.852217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.852225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.854348 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.854998 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.855000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.855000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.855057 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.855273 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.855274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.855275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.855406 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.855483 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.855691 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.855765 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:05:24.855917 12152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: Bootstrap starting.
I20260808 10:05:24.856669 12152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:24.857043 12152 log.cc:824] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:24.858556 12152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: No bootstrap required, opened a new log
I20260808 10:05:24.860241 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.845425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.24.162.188:38227)
0808 10:05:24.848795 (+ 3370us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.848797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.848805 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:05:24.848838 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.848839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.848850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.848923 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.849644 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.850366 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.854768 (+ 4402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.855103 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.855146 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.857010 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.857011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.857011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.857019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.858964 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.859363 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.859364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.859364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.859430 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.859662 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.859663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.859663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.859793 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.859883 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.860059 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.860144 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3348,"thread_start_us":78,"threads_started":1}
I20260808 10:05:24.860802 12152 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.860947 12152 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:24.860993 12152 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8eefedb3f64c419252e926c674c881, State: Initialized, Role: FOLLOWER
I20260808 10:05:24.861129 12152 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.861409 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.861482 12152 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:24.861543 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:05:24.863350 12331 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:05:24.863456 12331 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:05:24.863507 12064 master_runner.cc:186] Error getting master registration for 127.24.162.190:36125: OK, instance_id { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" instance_seqno: 1786183524438703 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:05:24.866200 12064 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:05:24.880966 12052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } attempt: 3
Aug 08 10:05:24 dist-test-slave-nfb5 krb5kdc[11932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:05:24.889948 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.881247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38227 (local address 127.0.0.1:36596)
0808 10:05:24.881298 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:24.881310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:24.881358 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:24.881509 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:24.881511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:24.881714 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:24.881767 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.881769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.882812 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.882813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.883719 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:24.883721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.884250 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.884251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:24.884322 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.884887 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:24.884892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:24.886593 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:24.888763 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.888764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.888764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.888982 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.889200 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:24.889200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:24.889201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:24.889260 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:24.889538 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:24.889539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:24.889587 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:05:24.889740 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 10:05:24.889842 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 10:05:24.890549 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:24.881305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.24.162.188:38227)
0808 10:05:24.881346 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:24.881348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:24.881357 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:24.881410 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:24.881411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:24.881422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:24.881485 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:24.881933 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.882723 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.883806 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:24.884169 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:24.884243 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:24.886851 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:24.886852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:24.886853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:24.886860 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:24.888666 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.889059 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.889060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.889060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.889130 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:24.889322 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:24.889323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:24.889324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:24.889451 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:24.889511 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:24.890365 (+ 854us) server_negotiation.cc:299] Negotiation successful
0808 10:05:24.890457 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:05:24.892515 12052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: Bootstrap starting.
I20260808 10:05:24.893303 12052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:24.893623 12052 log.cc:824] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:24.894590 12052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: No bootstrap required, opened a new log
I20260808 10:05:24.896624 12052 raft_consensus.cc:348] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.896780 12052 raft_consensus.cc:374] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:24.896816 12052 raft_consensus.cc:729] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a99ac375508c4a96b0261d84c2b004c8, State: Initialized, Role: FOLLOWER
I20260808 10:05:24.896958 12052 consensus_queue.cc:260] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.897287 12052 sys_catalog.cc:564] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:24.897253 12336 sys_catalog.cc:455] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.897450 12336 sys_catalog.cc:458] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:05:24.899111 12351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:05:24.899216 12351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:05:24.901252 11951 master_runner.cc:263] No leader master found from master a99ac375508c4a96b0261d84c2b004c8
I20260808 10:05:24.901307 11951 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 10:05:24.936553 11981 tablet.cc:2369] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:05:24.936601 12316 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:24.936728 12316 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.936987 12316 leader_election.cc:288] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a99ac375508c4a96b0261d84c2b004c8 (127.24.162.190:36125), bba4af5941094da6965f3994099a3f83 (127.24.162.188:38227)
I20260808 10:05:24.937355 12238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe8eefedb3f64c419252e926c674c881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba4af5941094da6965f3994099a3f83" is_pre_election: true
I20260808 10:05:24.937523 12238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe8eefedb3f64c419252e926c674c881 in term 0.
I20260808 10:05:24.937875 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe8eefedb3f64c419252e926c674c881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a99ac375508c4a96b0261d84c2b004c8" is_pre_election: true
I20260808 10:05:24.938026 12010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe8eefedb3f64c419252e926c674c881 in term 0.
I20260808 10:05:24.938218 12090 leader_election.cc:302] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a99ac375508c4a96b0261d84c2b004c8, fe8eefedb3f64c419252e926c674c881; no voters:
I20260808 10:05:24.938391 12316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:05:24.938445 12316 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:24.938475 12316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:24.940490 12316 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.940634 12316 leader_election.cc:288] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [CANDIDATE]: Term 1 election: Requested vote from peers a99ac375508c4a96b0261d84c2b004c8 (127.24.162.190:36125), bba4af5941094da6965f3994099a3f83 (127.24.162.188:38227)
I20260808 10:05:24.941520 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe8eefedb3f64c419252e926c674c881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a99ac375508c4a96b0261d84c2b004c8"
I20260808 10:05:24.941605 12010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:24.942528 12010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe8eefedb3f64c419252e926c674c881 in term 1.
I20260808 10:05:24.942642 12238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe8eefedb3f64c419252e926c674c881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba4af5941094da6965f3994099a3f83"
I20260808 10:05:24.942744 12238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:24.943058 12090 leader_election.cc:302] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a99ac375508c4a96b0261d84c2b004c8, fe8eefedb3f64c419252e926c674c881; no voters:
I20260808 10:05:24.943885 12238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe8eefedb3f64c419252e926c674c881 in term 1.
I20260808 10:05:24.944134 12316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:24.944325 12316 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [term 1 LEADER]: Becoming Leader. State: Replica: fe8eefedb3f64c419252e926c674c881, State: Running, Role: LEADER
I20260808 10:05:24.944429 12316 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } }
I20260808 10:05:24.944831 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8eefedb3f64c419252e926c674c881. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.944936 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:24.945235 12359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:24.945437 12359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:24.947450 12238 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [term 1 FOLLOWER]: Refusing update from remote peer fe8eefedb3f64c419252e926c674c881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:24.947777 12316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:24.949052 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8eefedb3f64c419252e926c674c881. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.949150 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:05:24.949527 12010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [term 1 FOLLOWER]: Refusing update from remote peer fe8eefedb3f64c419252e926c674c881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:24.949816 12316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:24.950973 12336 sys_catalog.cc:455] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8eefedb3f64c419252e926c674c881. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.951068 12336 sys_catalog.cc:458] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:05:24.952129 12360 mvcc.cc:205] Tried to move back new op lower bound from 7316207718182273024 to 7316207718172946432. Current Snapshot: MvccSnapshot[applied={T|T < 7316207718182273024}]
I20260808 10:05:24.952437 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.952520 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:24.952942 12288 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.953042 12288 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:05:24.954555 12362 mvcc.cc:205] Tried to move back new op lower bound from 7316207718182273024 to 7316207718172946432. Current Snapshot: MvccSnapshot[applied={T|T < 7316207718182273024}]
I20260808 10:05:24.955864 12359 catalog_manager.cc:1347] Generated new cluster ID: 36bcd0fa104b4be98ce244df927d88bf
I20260808 10:05:24.955927 12359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:05:24.964622 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.964746 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:24.965502 12336 sys_catalog.cc:455] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe8eefedb3f64c419252e926c674c881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a99ac375508c4a96b0261d84c2b004c8" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36125 } } peers { permanent_uuid: "fe8eefedb3f64c419252e926c674c881" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 43363 } } peers { permanent_uuid: "bba4af5941094da6965f3994099a3f83" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38227 } } }
I20260808 10:05:24.965605 12336 sys_catalog.cc:458] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:05:24.965713 12364 mvcc.cc:205] Tried to move back new op lower bound from 7316207718182273024 to 7316207718172946432. Current Snapshot: MvccSnapshot[applied={T|T < 7316207718182273024}]
I20260808 10:05:24.966162 12359 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:05:24.967805 12359 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:24.979825 12359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe8eefedb3f64c419252e926c674c881: Generated new TSK 0
I20260808 10:05:24.980010 12359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:05:25.846167 12309 catalog_manager.cc:1259] Loaded cluster ID: 36bcd0fa104b4be98ce244df927d88bf
I20260808 10:05:25.846251 12309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: loading cluster ID for follower catalog manager: success
I20260808 10:05:25.847121 12309 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: acquiring CA information for follower catalog manager: success
I20260808 10:05:25.847469 12309 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bba4af5941094da6965f3994099a3f83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:05:25.899727 12351 catalog_manager.cc:1259] Loaded cluster ID: 36bcd0fa104b4be98ce244df927d88bf
I20260808 10:05:25.899811 12351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: loading cluster ID for follower catalog manager: success
I20260808 10:05:25.900710 12351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: acquiring CA information for follower catalog manager: success
I20260808 10:05:25.901049 12351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a99ac375508c4a96b0261d84c2b004c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:05:26.033489 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 11951
I20260808 10:05:26.046298 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12064
I20260808 10:05:26.055652 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12170
2026-08-08T10:05:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3080 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:05:26 dist-test-slave-nfb5 krb5kdc[12385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:05:26 dist-test-slave-nfb5 krb5kdc[12385](info): set up 2 sockets
Aug 08 10:05:26 dist-test-slave-nfb5 krb5kdc[12385](info): commencing operation
krb5kdc: starting...
W20260808 10:05:27.286492 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:05:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:05:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:05:27.411911 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:42993
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:42993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:27.495391 12402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:27.495638 12402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:27.495678 12402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:27.496853 12402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:27.496923 12402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:27.496950 12402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:27.496984 12402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:27.497014 12402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:27.498512 12402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:42993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:42993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:27.498785 12402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:27.499089 12402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:27.504102 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:27.504123 12412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:27.504160 12413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:27.504247 12402 server_base.cc:1034] running on GCE node
I20260808 10:05:27.504482 12402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:27.504724 12402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:27.505872 12402 hybrid_clock.cc:648] HybridClock initialized: now 1786183527505848 us; error 39 us; skew 500 ppm
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183527, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:27.508492 12402 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:27.508982 12402 webserver.cc:460] Webserver started at http://127.24.162.190:40117/ using document root <none> and password file <none>
I20260808 10:05:27.509227 12402 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:27.509297 12402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:27.509459 12402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:27.510528 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "a97869fece00406c87f3176f341a95a3"
format_stamp: "Formatted at 2026-08-08 10:05:27 on dist-test-slave-nfb5"
server_key: "6fd0271b751be8c42372222dd73f64b4"
server_key_iv: "6029d9a0cd0a47fb53803ec1ca6def42"
server_key_version: "encryptionkey@0"
I20260808 10:05:27.510905 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "a97869fece00406c87f3176f341a95a3"
format_stamp: "Formatted at 2026-08-08 10:05:27 on dist-test-slave-nfb5"
server_key: "6fd0271b751be8c42372222dd73f64b4"
server_key_iv: "6029d9a0cd0a47fb53803ec1ca6def42"
server_key_version: "encryptionkey@0"
I20260808 10:05:27.512305 12402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:27.513054 12426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:27.513240 12402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:27.513343 12402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "a97869fece00406c87f3176f341a95a3"
format_stamp: "Formatted at 2026-08-08 10:05:27 on dist-test-slave-nfb5"
server_key: "6fd0271b751be8c42372222dd73f64b4"
server_key_iv: "6029d9a0cd0a47fb53803ec1ca6def42"
server_key_version: "encryptionkey@0"
I20260808 10:05:27.513427 12402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:27.530367 12402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:27.532445 12402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:27.532632 12402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:27.536919 12402 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:42993
I20260808 10:05:27.536945 12504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:42993 every 8 connection(s)
I20260808 10:05:27.537433 12402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:27.538370 12505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:27.540820 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3: Bootstrap starting.
I20260808 10:05:27.541431 12505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:27.541731 12505 log.cc:824] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:27.542579 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3: No bootstrap required, opened a new log
I20260808 10:05:27.544308 12505 raft_consensus.cc:348] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } }
I20260808 10:05:27.544482 12505 raft_consensus.cc:374] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:27.544531 12505 raft_consensus.cc:729] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97869fece00406c87f3176f341a95a3, State: Initialized, Role: FOLLOWER
I20260808 10:05:27.544730 12505 consensus_queue.cc:260] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } }
I20260808 10:05:27.544843 12505 raft_consensus.cc:388] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:27.544891 12505 raft_consensus.cc:482] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:27.544948 12505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:27.545508 12505 raft_consensus.cc:504] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } }
I20260808 10:05:27.545635 12505 leader_election.cc:302] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a97869fece00406c87f3176f341a95a3; no voters:
I20260808 10:05:27.545840 12505 leader_election.cc:288] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:05:27.545877 12508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:27.546067 12508 raft_consensus.cc:686] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [term 1 LEADER]: Becoming Leader. State: Replica: a97869fece00406c87f3176f341a95a3, State: Running, Role: LEADER
I20260808 10:05:27.546120 12505 sys_catalog.cc:564] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:27.546236 12508 consensus_queue.cc:237] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } }
I20260808 10:05:27.546622 12513 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a97869fece00406c87f3176f341a95a3. Latest consensus state: current_term: 1 leader_uuid: "a97869fece00406c87f3176f341a95a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } } }
I20260808 10:05:27.546757 12509 sys_catalog.cc:455] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a97869fece00406c87f3176f341a95a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a97869fece00406c87f3176f341a95a3" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 42993 } } }
I20260808 10:05:27.546854 12509 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:27.546818 12513 sys_catalog.cc:458] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:27.547231 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12402
I20260808 10:05:27.547343 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:05:27.547354 12520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:27.547538 25226 external_mini_cluster.cc:1428] Setting key 45fa0d315f31c2ee09580807fd154e9e
I20260808 10:05:27.547894 12520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:27.549655 12520 catalog_manager.cc:1347] Generated new cluster ID: 3d8cdba32b0140bd9cc6877584a69585
I20260808 10:05:27.549713 12520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:27.565572 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:27.548063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.24.162.190:42993)
0808 10:05:27.548259 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:27.548268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:27.548535 (+ 267us) server_negotiation.cc:407] Connection header received
0808 10:05:27.548660 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:27.548668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:27.548724 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:27.548863 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:27.555111 (+ 6248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:27.556583 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:27.557620 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:27.558268 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:27.558387 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:27.561545 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:27.561549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:27.561550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:27.561565 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:27.563714 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:27.564121 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:27.564125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:27.564125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:27.564243 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:27.564452 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:27.564453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:27.564453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:27.564786 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:27.564949 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:27.565053 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 10:05:27.565310 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260808 10:05:27.566607 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:05:27.568629 12520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:05:27.569147 12520 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:27.582826 12520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a97869fece00406c87f3176f341a95a3: Generated new TSK 0
I20260808 10:05:27.583070 12520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:05:27.620882 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12402
2026-08-08T10:05:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1566 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12542](info): set up 2 sockets
Aug 08 10:05:27 dist-test-slave-nfb5 krb5kdc[12542](info): commencing operation
krb5kdc: starting...
W20260808 10:05:28.842510 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:05:28 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:05:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:05:28.966552 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:41203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:29.049280 12567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:29.049495 12567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:29.049518 12567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:29.050678 12567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:29.050722 12567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:29.050737 12567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:29.050778 12567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:29.050809 12567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:29.052297 12567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:41203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:29.052557 12567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:29.052783 12567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:29.059209 12574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.059196 12573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.059198 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:29.059490 12567 server_base.cc:1034] running on GCE node
I20260808 10:05:29.059762 12567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:29.060052 12567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:29.061198 12567 hybrid_clock.cc:648] HybridClock initialized: now 1786183529061176 us; error 45 us; skew 500 ppm
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:29.063763 12567 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:29.064148 12567 webserver.cc:460] Webserver started at http://127.24.162.190:46341/ using document root <none> and password file <none>
I20260808 10:05:29.064330 12567 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:29.064370 12567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:29.064469 12567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:29.065477 12567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "1f03cb0a996f4060bf678550400a4002"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "1c3c7d5c805aa6de6c3a3f5e5f5f674d"
server_key_iv: "80125bac7d70f8649ca03596478be510"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.065793 12567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "1f03cb0a996f4060bf678550400a4002"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "1c3c7d5c805aa6de6c3a3f5e5f5f674d"
server_key_iv: "80125bac7d70f8649ca03596478be510"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.067140 12567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:05:29.067874 12592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.068053 12567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:29.068136 12567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1f03cb0a996f4060bf678550400a4002"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "1c3c7d5c805aa6de6c3a3f5e5f5f674d"
server_key_iv: "80125bac7d70f8649ca03596478be510"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.068190 12567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:29.076741 12567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:29.078758 12567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:29.078881 12567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:29.082748 12567 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:41203
I20260808 10:05:29.082805 12666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:41203 every 8 connection(s)
I20260808 10:05:29.083223 12567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:29.084282 12668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:29.086576 12668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap starting.
I20260808 10:05:29.087200 12668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:29.087504 12668 log.cc:824] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:29.088236 12668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: No bootstrap required, opened a new log
I20260808 10:05:29.089823 12668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:29.089982 12668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:29.090034 12668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Initialized, Role: FOLLOWER
I20260808 10:05:29.090205 12668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:29.090288 12668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:29.090329 12668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:29.090381 12668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:29.091245 12668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:29.091384 12668 leader_election.cc:302] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f03cb0a996f4060bf678550400a4002; no voters:
I20260808 10:05:29.091557 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12567
I20260808 10:05:29.091584 12668 leader_election.cc:288] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:05:29.091612 12672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:29.091665 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:05:29.091842 12672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 LEADER]: Becoming Leader. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Running, Role: LEADER
I20260808 10:05:29.091884 25226 external_mini_cluster.cc:1428] Setting key 36165776aa708cf44610157475754d67
I20260808 10:05:29.091962 12672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:29.092012 12668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:29.092344 12673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:29.092361 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f03cb0a996f4060bf678550400a4002. Latest consensus state: current_term: 1 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:29.092447 12673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:29.092468 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:29.092880 12681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:29.093425 12681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:29.095949 12681 catalog_manager.cc:1347] Generated new cluster ID: b6011d99454748ed8f59dc50075acfb5
I20260808 10:05:29.096004 12681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:29.103273 12681 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:05:29.104065 12681 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:29.104089 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.092504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.24.162.190:41203)
0808 10:05:29.093330 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.093339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.093371 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:05:29.093489 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.093496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.093554 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.094448 (+ 894us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:29.094519 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.095503 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.096547 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.097157 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.097262 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.100142 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:29.100147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:29.100147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:29.100159 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:29.102316 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.102728 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.102731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.102732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.102845 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.103113 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.103114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.103114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.103434 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:29.103605 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.103714 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.103853 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":80,"threads_started":1}
I20260808 10:05:29.113854 12681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Generated new TSK 0
I20260808 10:05:29.114095 12681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:05:29.145833 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:29.229681 12700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:29.229895 12700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:29.229919 12700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:29.231108 12700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:29.231148 12700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:29.231175 12700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:29.232581 12700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:29.232779 12700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:29.232959 12700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:29.238118 12707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.238118 12710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.238158 12708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:29.238538 12700 server_base.cc:1034] running on GCE node
I20260808 10:05:29.238718 12700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:29.238977 12700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:29.240139 12700 hybrid_clock.cc:648] HybridClock initialized: now 1786183529240111 us; error 44 us; skew 500 ppm
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:29.242887 12700 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:29.243388 12700 webserver.cc:460] Webserver started at http://127.24.162.129:41799/ using document root <none> and password file <none>
I20260808 10:05:29.243649 12700 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:29.243695 12700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:29.243829 12700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:29.244913 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "69e44231bdd547a58131a66be62f5e0a"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "bc96c77c026b77ae2605c0acc8c92f53"
server_key_iv: "1ba76399d66d237eb1887a7ea00f049b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.245285 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "69e44231bdd547a58131a66be62f5e0a"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "bc96c77c026b77ae2605c0acc8c92f53"
server_key_iv: "1ba76399d66d237eb1887a7ea00f049b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.246774 12700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:05:29.247665 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.247905 12700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:29.248010 12700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "69e44231bdd547a58131a66be62f5e0a"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "bc96c77c026b77ae2605c0acc8c92f53"
server_key_iv: "1ba76399d66d237eb1887a7ea00f049b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.248093 12700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:29.257232 12700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:29.259565 12700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:29.259761 12700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:29.260001 12700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:29.260318 12700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:29.260375 12700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.260417 12700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:29.260442 12700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.266492 12700 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41795
I20260808 10:05:29.266537 12876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41795 every 8 connection(s)
I20260808 10:05:29.266851 12700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:29.271158 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12700
I20260808 10:05:29.271275 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:05:29.271481 25226 external_mini_cluster.cc:1428] Setting key 96bced5628415d840c2fea86e2e30579
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:29.277137 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.267624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:37867 (local address 127.24.162.190:41203)
0808 10:05:29.267715 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.267718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.267899 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:05:29.268383 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.268384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.268399 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.268448 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:29.269367 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.269899 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.271109 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.271459 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.271549 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.274079 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:29.274080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:29.274080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:29.274088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:29.275630 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.276086 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.276087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.276087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.276153 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.276469 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.276470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.276471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.276600 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:29.276643 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.276946 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.277045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:05:29.277348 12880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.267576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:37867)
0808 10:05:29.267775 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.267812 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.268299 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.268509 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.268512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.269080 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:29.269282 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.269287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.269987 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.269988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.271019 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.271022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.271580 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.271582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.271743 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.272364 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:29.272372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:29.273965 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:29.275741 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.275750 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.275752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.275990 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.276231 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.276232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.276232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.276357 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.276682 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:29.276688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:29.276809 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.276922 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.277087 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
I20260808 10:05:29.277908 12877 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:29.277998 12877 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:29.278203 12877 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:29.278863 12616 ts_manager.cc:194] Registered new tserver with Master: 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795)
I20260808 10:05:29.279863 12616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:37867
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:05:29.297928 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:29.380288 12883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:29.380483 12883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:29.380507 12883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:29.381582 12883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:29.381618 12883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:29.381644 12883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:29.383073 12883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:29.383256 12883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:29.383430 12883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:29.388415 12893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.388389 12894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:29.388574 12883 server_base.cc:1034] running on GCE node
W20260808 10:05:29.388389 12897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:29.388850 12883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:29.389057 12883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:29.390195 12883 hybrid_clock.cc:648] HybridClock initialized: now 1786183529390183 us; error 29 us; skew 500 ppm
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:29.393002 12883 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:29.393471 12883 webserver.cc:460] Webserver started at http://127.24.162.130:33203/ using document root <none> and password file <none>
I20260808 10:05:29.393718 12883 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:29.393790 12883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:29.393952 12883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:29.395001 12883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "160ee425755422d814392520643e3326"
server_key_iv: "7978effb11bc373cd5ae71619d7fbe4b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.395319 12883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "160ee425755422d814392520643e3326"
server_key_iv: "7978effb11bc373cd5ae71619d7fbe4b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.396559 12883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:29.397251 12906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.397428 12883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:05:29.397509 12883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "160ee425755422d814392520643e3326"
server_key_iv: "7978effb11bc373cd5ae71619d7fbe4b"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.397562 12883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:29.410144 12883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:29.412009 12883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:29.412117 12883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:29.412292 12883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:29.412534 12883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:29.412592 12883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.412626 12883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:29.412662 12883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.418375 12883 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40369
I20260808 10:05:29.418452 13067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40369 every 8 connection(s)
I20260808 10:05:29.418790 12883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:29.422860 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 12883
I20260808 10:05:29.423007 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:05:29.423243 25226 external_mini_cluster.cc:1428] Setting key 3c24ce0f5f7e08f23e130f0a4e14190c
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:29.429212 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.419587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:42701 (local address 127.24.162.190:41203)
0808 10:05:29.419677 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.419679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.419859 (+ 180us) server_negotiation.cc:407] Connection header received
0808 10:05:29.420403 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.420405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.420417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.420473 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:29.421432 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.421956 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.422956 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.423248 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.423345 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.425894 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:29.425895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:29.425896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:29.425905 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:29.427581 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.427978 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.427979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.427979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.428045 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.428319 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.428319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.428320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.428479 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:29.428559 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.428988 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.429092 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:05:29.429224 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.419539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:42701)
0808 10:05:29.419724 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.419771 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.420280 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.420538 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.420542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.421134 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:29.421324 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.421328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.422055 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.422056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.422842 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.422844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.423355 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.423356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.423486 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.424029 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:29.424037 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:29.425782 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:29.427666 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.427672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.427673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.427899 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.428132 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.428133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.428133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.428241 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.428669 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:29.428676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:29.428843 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.428933 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.429052 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260808 10:05:29.429858 13068 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:29.429972 13068 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:29.430186 13068 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:29.430554 12616 ts_manager.cc:194] Registered new tserver with Master: 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
I20260808 10:05:29.431012 12616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:42701
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:05:29.453301 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:29.537351 13075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:29.537561 13075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:29.537585 13075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:29.538853 13075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:29.538900 13075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:29.538991 13075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:29.540474 13075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:29.540752 13075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:29.541002 13075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:29.546252 13086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.546267 13088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:29.546304 13085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:29.546541 13075 server_base.cc:1034] running on GCE node
I20260808 10:05:29.546753 13075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:29.547004 13075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:29.548158 13075 hybrid_clock.cc:648] HybridClock initialized: now 1786183529548118 us; error 60 us; skew 500 ppm
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:29.550724 13075 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:29.551229 13075 webserver.cc:460] Webserver started at http://127.24.162.131:42191/ using document root <none> and password file <none>
I20260808 10:05:29.551482 13075 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:29.551528 13075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:29.551678 13075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:29.552734 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "1de46d62b4924a8d95dbfc143e5f8204"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "5e1d507b057cb5cf313a7fe21621f13a"
server_key_iv: "e935378e92f4bafa4dc889bf615a87d8"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.553099 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "1de46d62b4924a8d95dbfc143e5f8204"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "5e1d507b057cb5cf313a7fe21621f13a"
server_key_iv: "e935378e92f4bafa4dc889bf615a87d8"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.554415 13075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:05:29.555235 13097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.555419 13075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:29.555513 13075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "1de46d62b4924a8d95dbfc143e5f8204"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "5e1d507b057cb5cf313a7fe21621f13a"
server_key_iv: "e935378e92f4bafa4dc889bf615a87d8"
server_key_version: "encryptionkey@0"
I20260808 10:05:29.555588 13075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:29.578207 13075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:29.580678 13075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:29.580857 13075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:29.581099 13075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:29.581434 13075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:29.581490 13075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.581534 13075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:29.581562 13075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:29.587340 13075 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33239
I20260808 10:05:29.587437 13255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33239 every 8 connection(s)
I20260808 10:05:29.587744 13075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:05:29.588992 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 13075
I20260808 10:05:29.589074 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:05:29.589283 25226 external_mini_cluster.cc:1428] Setting key 74377a512f569fe51b1055c83c0bdb10
Aug 08 10:05:29 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183529, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:29.597483 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.588647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:60949 (local address 127.24.162.190:41203)
0808 10:05:29.588692 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.588694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.588894 (+ 200us) server_negotiation.cc:407] Connection header received
0808 10:05:29.589503 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.589505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.589516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.589620 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:29.590765 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.591376 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.592509 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.592745 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.592818 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.595129 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:29.595129 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:29.595130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:29.595136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:29.596332 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.596689 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.596690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.596690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.596740 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.596951 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.596952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.596952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.597051 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:29.597102 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.597345 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.597421 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:05:29.597640 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.588598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:60949)
0808 10:05:29.588758 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.588798 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.589385 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.589702 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.589707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.590489 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:29.590676 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.590680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.591457 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.591458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.592417 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.592419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.592834 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.592835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.592936 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.593499 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:29.593513 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:29.595050 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:29.596404 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.596410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.596411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.596624 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.596799 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:29.596799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:29.596799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:29.596895 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:29.597107 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:29.597111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:29.597224 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.597306 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.597431 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":102,"threads_started":1}
I20260808 10:05:29.598191 13257 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:29.598285 13257 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:29.598424 13257 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:29.598803 12616 ts_manager.cc:194] Registered new tserver with Master: 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239)
I20260808 10:05:29.599206 12616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:60949
I20260808 10:05:29.601032 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:05:29.608224 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.602461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.24.162.190:41203)
0808 10:05:29.602517 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.602519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.602612 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:05:29.602650 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.602650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.602659 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.602714 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:29.603301 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.603766 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.604545 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.604723 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.604777 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.605860 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:29.605861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:29.605862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:29.605868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:29.607284 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.607585 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.607586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.607586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.607636 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:29.607798 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:29.607798 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:29.607798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:29.607934 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:29.608008 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.608095 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.608151 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:05:29.609791 12616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:05:29.611542 12616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:05:29.617852 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.614131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.24.162.130:40369)
0808 10:05:29.614333 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.614338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.614351 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:05:29.614391 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.614393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.614410 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.614484 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:29.614662 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.615520 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.616690 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.617437 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.617510 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.617564 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.617727 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.617769 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260808 10:05:29.617867 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.613672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40369 (local address 127.0.0.1:47820)
0808 10:05:29.614002 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.614013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.614072 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.614505 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.614508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.614511 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:29.614575 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.614577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.615617 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.615618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.616600 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.616603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.617529 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.617530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.617606 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.617621 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.617711 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.617766 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":74,"threads_started":1}
I20260808 10:05:29.618098 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.613653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33239 (local address 127.0.0.1:46180)
0808 10:05:29.613934 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.613976 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.614031 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.615042 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.615043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.615046 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:29.615090 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.615091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.616282 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.616283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.617099 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.617101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.617881 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.617882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.617969 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.617973 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.618005 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.618039 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":66,"threads_started":1}
I20260808 10:05:29.618409 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.613725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.24.162.131:33239)
0808 10:05:29.614819 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.614824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.614833 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:29.614867 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.614869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.614883 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.615023 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:29.615167 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.616168 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.617201 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.617787 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.618176 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.618249 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.618277 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.618320 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":144,"threads_started":1}
I20260808 10:05:29.618418 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.613651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.0.0.1:37810)
0808 10:05:29.613919 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.613960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.614031 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.614609 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.614610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.614612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:29.614681 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.614683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.616019 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.616020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.617212 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.617215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.618153 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.618154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.618227 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.618231 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.618281 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.618335 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":26,"thread_start_us":62,"threads_started":1}
I20260808 10:05:29.619040 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.613926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.24.162.129:41795)
0808 10:05:29.614366 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.614372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.614384 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:05:29.614424 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.614427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.614444 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.614522 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:29.614742 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.615882 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.617309 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.618063 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.618152 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.618203 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.618891 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.618960 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":90,"threads_started":1}
I20260808 10:05:29.619082 12974 tablet_service.cc:1467] Processing CreateTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 (DEFAULT_TABLE table=test-table [id=b1720b9a872249f4bf156218cde49659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:29.619145 13158 tablet_service.cc:1467] Processing CreateTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 (DEFAULT_TABLE table=test-table [id=b1720b9a872249f4bf156218cde49659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:29.619493 13158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7a40ecbc4b4aee93325dca2b129aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:29.619501 12974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7a40ecbc4b4aee93325dca2b129aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:29.619668 12788 tablet_service.cc:1467] Processing CreateTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 (DEFAULT_TABLE table=test-table [id=b1720b9a872249f4bf156218cde49659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:29.619971 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7a40ecbc4b4aee93325dca2b129aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:29.622030 13278 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Bootstrap starting.
I20260808 10:05:29.622632 13278 tablet_bootstrap.cc:654] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:29.622809 13279 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Bootstrap starting.
I20260808 10:05:29.623175 13280 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Bootstrap starting.
I20260808 10:05:29.623266 13278 log.cc:824] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:29.623540 13279 tablet_bootstrap.cc:654] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:29.623824 13280 tablet_bootstrap.cc:654] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:29.624049 13279 log.cc:824] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:29.624058 13278 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: No bootstrap required, opened a new log
I20260808 10:05:29.624130 13278 ts_tablet_manager.cc:1397] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:29.624440 13280 log.cc:824] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:29.624722 13279 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: No bootstrap required, opened a new log
I20260808 10:05:29.624786 13279 ts_tablet_manager.cc:1397] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:05:29.625290 13280 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: No bootstrap required, opened a new log
I20260808 10:05:29.625360 13280 ts_tablet_manager.cc:1397] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:29.625808 13278 raft_consensus.cc:348] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.625928 13278 raft_consensus.cc:374] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:29.625949 13278 raft_consensus.cc:729] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de46d62b4924a8d95dbfc143e5f8204, State: Initialized, Role: FOLLOWER
I20260808 10:05:29.626067 13278 consensus_queue.cc:260] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.626349 13278 ts_tablet_manager.cc:1428] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:29.626365 13257 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
I20260808 10:05:29.627239 13279 raft_consensus.cc:348] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.627305 13280 raft_consensus.cc:348] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.627408 13279 raft_consensus.cc:374] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:29.627449 13280 raft_consensus.cc:374] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:29.627446 13279 raft_consensus.cc:729] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbe8cb5eeb143ba8403b1821611abc1, State: Initialized, Role: FOLLOWER
I20260808 10:05:29.627483 13280 raft_consensus.cc:729] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Initialized, Role: FOLLOWER
I20260808 10:05:29.627579 13280 consensus_queue.cc:260] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.627593 13279 consensus_queue.cc:260] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.627866 13279 ts_tablet_manager.cc:1428] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 10:05:29.627866 13280 ts_tablet_manager.cc:1428] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:05:29.627895 13068 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
I20260808 10:05:29.628002 12877 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
W20260808 10:05:29.669404 13070 tablet.cc:2369] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:05:29.767601 12879 tablet.cc:2369] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:05:29.812376 13288 raft_consensus.cc:482] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:29.812546 13288 raft_consensus.cc:504] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.812950 13288 leader_election.cc:288] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
I20260808 10:05:29.817206 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.813308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:41259 (local address 127.24.162.130:40369)
0808 10:05:29.813366 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.813368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.813380 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:05:29.813463 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.813464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.813475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.813553 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:29.813851 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.814609 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816191 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.816796 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816874 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.816883 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.817091 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.817130 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:05:29.817206 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.813250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:35921 (local address 127.24.162.131:33239)
0808 10:05:29.813302 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.813305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.813382 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:05:29.813466 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.813467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.813477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.813553 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:29.813858 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.814604 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816191 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.816786 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816874 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.816883 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.817090 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.817130 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:05:29.817282 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.813170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33239 (local address 127.24.162.129:35921)
0808 10:05:29.813306 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.813319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.813396 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.813580 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.813581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.813585 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:29.813758 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.813763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.814737 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.814738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.816092 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.816095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.816901 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.816996 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.817003 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.817070 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.817133 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":67,"thread_start_us":66,"threads_started":1}
I20260808 10:05:29.817286 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.813170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40369 (local address 127.24.162.129:41259)
0808 10:05:29.813335 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.813345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:29.813396 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:29.813580 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:29.813581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:29.813584 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:29.813758 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.813761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.814710 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.814712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.816092 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:29.816095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.816903 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.816904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:29.816996 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.817005 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:05:29.817070 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:05:29.817133 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":103,"thread_start_us":97,"threads_started":1}
I20260808 10:05:29.817723 13187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7a40ecbc4b4aee93325dca2b129aa4" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de46d62b4924a8d95dbfc143e5f8204" is_pre_election: true
I20260808 10:05:29.817728 13005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7a40ecbc4b4aee93325dca2b129aa4" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" is_pre_election: true
I20260808 10:05:29.817906 13187 raft_consensus.cc:2454] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 0.
I20260808 10:05:29.817916 13005 raft_consensus.cc:2454] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 0.
I20260808 10:05:29.818185 12724 leader_election.cc:302] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de46d62b4924a8d95dbfc143e5f8204, 69e44231bdd547a58131a66be62f5e0a; no voters:
I20260808 10:05:29.818336 13288 raft_consensus.cc:2781] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:05:29.818413 13288 raft_consensus.cc:482] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:29.818432 13288 raft_consensus.cc:3037] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:29.819238 13288 raft_consensus.cc:504] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.819398 13288 leader_election.cc:288] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 1 election: Requested vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
I20260808 10:05:29.819592 13187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7a40ecbc4b4aee93325dca2b129aa4" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de46d62b4924a8d95dbfc143e5f8204"
I20260808 10:05:29.819612 13005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7a40ecbc4b4aee93325dca2b129aa4" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
I20260808 10:05:29.819696 13187 raft_consensus.cc:3037] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:29.819692 13005 raft_consensus.cc:3037] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:29.820425 13005 raft_consensus.cc:2454] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 1.
I20260808 10:05:29.820552 13187 raft_consensus.cc:2454] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 1.
I20260808 10:05:29.820618 12726 leader_election.cc:302] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbe8cb5eeb143ba8403b1821611abc1, 69e44231bdd547a58131a66be62f5e0a; no voters:
I20260808 10:05:29.820772 13288 raft_consensus.cc:2781] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:29.820955 13288 raft_consensus.cc:686] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 LEADER]: Becoming Leader. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Running, Role: LEADER
I20260808 10:05:29.821059 13288 consensus_queue.cc:237] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:29.821939 12616 catalog_manager.cc:5495] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 69e44231bdd547a58131a66be62f5e0a (127.24.162.129). New cstate: current_term: 1 leader_uuid: "69e44231bdd547a58131a66be62f5e0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } health_report { overall_health: UNKNOWN } } }
W20260808 10:05:29.838289 13258 tablet.cc:2369] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:05:29.843642 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.841209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.24.162.129:41795)
0808 10:05:29.841274 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:29.841277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:29.841285 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:05:29.841334 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:29.841335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:29.841352 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:29.841432 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:05:29.841575 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.842051 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.842818 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:29.843037 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:29.843120 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:29.843297 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:05:29.843399 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:05:29.843486 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:29.843531 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 10:05:29.843552 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:05:29.845577 13187 raft_consensus.cc:1261] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Refusing update from remote peer 69e44231bdd547a58131a66be62f5e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:29.845577 13005 raft_consensus.cc:1261] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Refusing update from remote peer 69e44231bdd547a58131a66be62f5e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:29.845961 13292 consensus_queue.cc:1035] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:29.846143 13288 consensus_queue.cc:1035] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:29.849442 13300 mvcc.cc:205] Tried to move back new op lower bound from 7316207738244366336 to 7316207738147770368. Current Snapshot: MvccSnapshot[applied={T|T < 7316207738244366336}]
I20260808 10:05:29.851907 12615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:29.852017 12615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:29.852851 12615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Sending DeleteTablet for 3 replicas of tablet ef7a40ecbc4b4aee93325dca2b129aa4
I20260808 10:05:29.853201 13158 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:29 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46180
I20260808 10:05:29.853225 12974 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:29 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:47820
I20260808 10:05:29.853209 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:29 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:37810
I20260808 10:05:29.853473 13314 tablet_replica.cc:330] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: stopping tablet replica
I20260808 10:05:29.853581 13314 raft_consensus.cc:2229] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:29.853585 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12700
I20260808 10:05:29.853596 13313 tablet_replica.cc:330] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: stopping tablet replica
I20260808 10:05:29.853786 13313 raft_consensus.cc:2229] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:05:29.853969 13313 raft_consensus.cc:2258] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:29.862143 13314 raft_consensus.cc:2258] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:29.862840 13310 tablet_replica.cc:330] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: stopping tablet replica
I20260808 10:05:29.863014 13310 raft_consensus.cc:2229] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:29.863132 13314 ts_tablet_manager.cc:1905] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:29.863162 13310 raft_consensus.cc:2258] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:29.863654 13310 ts_tablet_manager.cc:1905] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:29.865150 13314 ts_tablet_manager.cc:1918] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:29.865264 13314 log.cc:1196] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/ef7a40ecbc4b4aee93325dca2b129aa4
I20260808 10:05:29.865255 13310 ts_tablet_manager.cc:1918] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:29.865346 13310 log.cc:1196] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/ef7a40ecbc4b4aee93325dca2b129aa4
I20260808 10:05:29.865628 13310 ts_tablet_manager.cc:1939] T ef7a40ecbc4b4aee93325dca2b129aa4 P 1de46d62b4924a8d95dbfc143e5f8204: Deleting consensus metadata
I20260808 10:05:29.865648 13314 ts_tablet_manager.cc:1939] T ef7a40ecbc4b4aee93325dca2b129aa4 P 2dbe8cb5eeb143ba8403b1821611abc1: Deleting consensus metadata
I20260808 10:05:29.866134 12596 catalog_manager.cc:4841] TS 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239): tablet ef7a40ecbc4b4aee93325dca2b129aa4 (table test-table [id=b1720b9a872249f4bf156218cde49659]) successfully deleted
I20260808 10:05:29.866139 12600 catalog_manager.cc:4841] TS 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369): tablet ef7a40ecbc4b4aee93325dca2b129aa4 (table test-table [id=b1720b9a872249f4bf156218cde49659]) successfully deleted
W20260808 10:05:29.872063 12599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:41795 (error 108)
I20260808 10:05:29.872630 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:29.872480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.0.0.1:37822)
0808 10:05:29.872536 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:29.872574 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:05:29.872705 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12883
W20260808 10:05:29.872826 12599 catalog_manager.cc:4584] TS 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef7a40ecbc4b4aee93325dca2b129aa4: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41795: connect: Connection refused (error 111)
I20260808 10:05:29.883478 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 13075
I20260808 10:05:29.893597 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 12567
I20260808 10:05:29.905412 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--webserver_interface=127.24.162.190
--webserver_port=46341
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:41203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:29.988773 13316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:29.988965 13316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:29.988987 13316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:29.990041 13316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:29.990077 13316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:29.990090 13316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:29.990103 13316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:29.990115 13316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:29.991551 13316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:41203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:29.991775 13316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:29.991959 13316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.001703 13327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.001865 13316 server_base.cc:1034] running on GCE node
W20260808 10:05:30.001748 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.001916 13325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.002261 13316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.002566 13316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.003719 13316 hybrid_clock.cc:648] HybridClock initialized: now 1786183530003700 us; error 34 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.009605 13316 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:30.010126 13316 webserver.cc:460] Webserver started at http://127.24.162.190:46341/ using document root <none> and password file <none>
I20260808 10:05:30.010352 13316 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.010411 13316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.012007 13316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.012897 13336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.013125 13316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.013236 13316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1f03cb0a996f4060bf678550400a4002"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "1c3c7d5c805aa6de6c3a3f5e5f5f674d"
server_key_iv: "80125bac7d70f8649ca03596478be510"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.013695 13316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.024923 13316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.027422 13316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.027585 13316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.031968 13316 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:41203
I20260808 10:05:30.032008 13421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:41203 every 8 connection(s)
I20260808 10:05:30.032413 13316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:30.034107 13423 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:05:30.034792 13423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap starting.
I20260808 10:05:30.041133 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 13316
I20260808 10:05:30.041499 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:41795
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=41799
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:30.044366 13423 log.cc:824] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.046594 13423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:30.046980 13423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap complete.
I20260808 10:05:30.049494 13423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.049721 13423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.049896 13423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.049964 13423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:30.050015 13423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:30.050072 13423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:05:30.051215 13423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.051308 13423 leader_election.cc:302] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f03cb0a996f4060bf678550400a4002; no voters:
I20260808 10:05:30.051544 13423 leader_election.cc:288] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:05:30.051580 13428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:05:30.051692 13428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 LEADER]: Becoming Leader. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Running, Role: LEADER
I20260808 10:05:30.051788 13428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.051843 13423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:30.052080 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f03cb0a996f4060bf678550400a4002. Latest consensus state: current_term: 2 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:30.052155 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:30.052086 13429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:30.052209 13429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:30.052440 13435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:30.053334 13435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:30.053443 13435 catalog_manager.cc:1259] Loaded cluster ID: b6011d99454748ed8f59dc50075acfb5
I20260808 10:05:30.053473 13435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:05:30.053725 13435 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 10:05:30.058352 13435 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 10:05:30.060901 13435 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:30.061103 13435 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Loaded TSK: 0
I20260808 10:05:30.061336 13435 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:05:30.126874 13425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:30.127218 13425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:30.127257 13425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:30.128373 13425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:30.128438 13425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:30.128484 13425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:30.129920 13425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:41795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=41799
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:30.130190 13425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:30.130438 13425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.136463 13466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.136435 13463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.136452 13464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.137748 13425 server_base.cc:1034] running on GCE node
I20260808 10:05:30.137970 13425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.138253 13425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.139427 13425 hybrid_clock.cc:648] HybridClock initialized: now 1786183530139403 us; error 44 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.142163 13425 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:30.142675 13425 webserver.cc:460] Webserver started at http://127.24.162.129:41799/ using document root <none> and password file <none>
I20260808 10:05:30.142992 13425 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.143081 13425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.144503 13425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:30.145227 13475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.145406 13425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:30.145505 13425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "69e44231bdd547a58131a66be62f5e0a"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "bc96c77c026b77ae2605c0acc8c92f53"
server_key_iv: "1ba76399d66d237eb1887a7ea00f049b"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.145849 13425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.155221 13425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.157418 13425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.157598 13425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.157829 13425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:30.158275 13485 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:05:30.160158 13425 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:05:30.160231 13425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:05:30.160272 13425 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:05:30.160924 13425 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:05:30.160982 13425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:30.161044 13485 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Bootstrap starting.
I20260808 10:05:30.168286 13425 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41795
I20260808 10:05:30.168648 13425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:30.172137 13485 log.cc:824] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.173130 13630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41795 every 8 connection(s)
I20260808 10:05:30.174664 13485 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:30.175014 13485 tablet_bootstrap.cc:492] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Bootstrap complete.
I20260808 10:05:30.176257 13485 ts_tablet_manager.cc:1397] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260808 10:05:30.177546 13485 raft_consensus.cc:348] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:30.177836 13485 raft_consensus.cc:729] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.177907 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 13425
I20260808 10:05:30.177994 13485 consensus_queue.cc:260] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } } peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } }
I20260808 10:05:30.178203 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:40369
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=33203
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:30.178242 13485 ts_tablet_manager.cc:1428] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:30.184917 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.169380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:59097)
0808 10:05:30.172388 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.172435 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.172955 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.173782 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.173786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.174069 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:30.174265 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.174270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.175588 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.175589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.176449 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.176452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.177056 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.177057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.177193 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.177749 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:30.177771 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:30.179735 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:30.182880 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.182889 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.182892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.183282 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.183621 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.183622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.183623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.183766 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.184277 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:30.184284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:30.184449 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.184553 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.184714 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2956,"thread_start_us":99,"threads_started":1}
I20260808 10:05:30.185431 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.169441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59097 (local address 127.24.162.190:41203)
0808 10:05:30.173414 (+ 3973us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.173421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.173449 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:05:30.173551 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.173556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.173596 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.173773 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:30.174389 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.175486 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.176539 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.176940 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.177073 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.179884 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:30.179890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:30.179890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:30.179904 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:30.182769 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.183399 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.183403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.183403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.183534 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.183857 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.183858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.183859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.184184 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:30.184288 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.184662 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.184826 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3926,"thread_start_us":81,"threads_started":1}
I20260808 10:05:30.186111 13631 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:30.186218 13631 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:30.186431 13631 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:30.187228 13369 ts_manager.cc:194] Registered new tserver with Master: 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795)
I20260808 10:05:30.188400 13369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:59097
I20260808 10:05:30.189429 13631 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
I20260808 10:05:30.194645 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.187861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.0.0.1:37836)
0808 10:05:30.191049 (+ 3188us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.191086 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.191153 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.191450 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.191454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.191456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:30.191519 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.191521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.192763 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.192763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.193533 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.193535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.194429 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.194430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.194491 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.194502 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.194554 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.194593 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3152,"thread_start_us":91,"threads_started":1}
I20260808 10:05:30.194690 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.188157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37836 (local address 127.24.162.129:41795)
0808 10:05:30.191289 (+ 3132us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.191293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.191302 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:05:30.191332 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.191334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.191348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.191420 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:30.191604 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.192677 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.193611 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.194340 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.194407 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.194455 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.194585 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.194625 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3102,"thread_start_us":99,"threads_started":1}
I20260808 10:05:30.195087 13537 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7a40ecbc4b4aee93325dca2b129aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:29 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:37836
I20260808 10:05:30.195602 13641 tablet_replica.cc:330] stopping tablet replica
I20260808 10:05:30.195709 13641 raft_consensus.cc:2229] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:30.195796 13641 raft_consensus.cc:2258] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:30.196113 13641 ts_tablet_manager.cc:1905] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:30.198138 13641 ts_tablet_manager.cc:1918] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:30.198212 13641 log.cc:1196] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/ef7a40ecbc4b4aee93325dca2b129aa4
I20260808 10:05:30.198444 13641 ts_tablet_manager.cc:1939] T ef7a40ecbc4b4aee93325dca2b129aa4 P 69e44231bdd547a58131a66be62f5e0a: Deleting consensus metadata
I20260808 10:05:30.198983 13345 catalog_manager.cc:4841] TS 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795): tablet ef7a40ecbc4b4aee93325dca2b129aa4 (table test-table [id=b1720b9a872249f4bf156218cde49659]) successfully deleted
W20260808 10:05:30.266010 13637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:30.266263 13637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:30.266304 13637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:30.267470 13637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:30.267539 13637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:30.267582 13637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:30.269030 13637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:40369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=33203
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:30.269292 13637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:30.269528 13637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.274477 13648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.274525 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.274466 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.274648 13637 server_base.cc:1034] running on GCE node
I20260808 10:05:30.275156 13637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.275398 13637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.276567 13637 hybrid_clock.cc:648] HybridClock initialized: now 1786183530276558 us; error 41 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.279178 13637 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:30.279662 13637 webserver.cc:460] Webserver started at http://127.24.162.130:33203/ using document root <none> and password file <none>
I20260808 10:05:30.279902 13637 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.279985 13637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.281387 13637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:30.282027 13658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.282199 13637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:05:30.282302 13637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "160ee425755422d814392520643e3326"
server_key_iv: "7978effb11bc373cd5ae71619d7fbe4b"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.282644 13637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.292884 13637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.294795 13637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.294987 13637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.295223 13637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:30.295521 13637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:30.295576 13637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.295619 13637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:30.295646 13637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.301443 13637 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40369
I20260808 10:05:30.301515 13815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40369 every 8 connection(s)
I20260808 10:05:30.301822 13637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:30.304764 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 13637
I20260808 10:05:30.305063 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:33239
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=42191
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:30.311561 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.302582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:57739 (local address 127.24.162.190:41203)
0808 10:05:30.302631 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.302634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.302902 (+ 268us) server_negotiation.cc:407] Connection header received
0808 10:05:30.303409 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.303410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.303423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.303471 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:30.304346 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.304837 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.306033 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.306370 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.306452 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.308561 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:30.308562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:30.308563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:30.308571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:30.310061 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.310494 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.310496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.310496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.310563 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.310821 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.310822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.310823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.311000 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:30.311083 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.311367 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.311468 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:05:30.311707 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.302598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:57739)
0808 10:05:30.302780 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.302820 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.303321 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.303526 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.303529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.304090 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:30.304267 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.304271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.304972 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.304973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.305872 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.305875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.306464 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.306464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.306564 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.307109 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:30.307117 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:30.308456 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:30.310176 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.310183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.310186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.310410 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.310634 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.310635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.310635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.310749 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.311074 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:30.311081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:30.311224 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.311331 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.311485 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260808 10:05:30.312153 13816 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:30.312270 13816 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:30.312489 13816 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:30.312915 13369 ts_manager.cc:194] Registered new tserver with Master: 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
I20260808 10:05:30.313303 13369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:57739
W20260808 10:05:30.388933 13822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:30.389164 13822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:30.389194 13822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:30.390553 13822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:30.390599 13822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:30.390630 13822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:30.392305 13822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:33239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=42191
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:30.392570 13822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:30.392823 13822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.398254 13828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.398255 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.398293 13833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.398518 13822 server_base.cc:1034] running on GCE node
I20260808 10:05:30.398758 13822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.399032 13822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.400210 13822 hybrid_clock.cc:648] HybridClock initialized: now 1786183530400198 us; error 28 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.402951 13822 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:30.403395 13822 webserver.cc:460] Webserver started at http://127.24.162.131:42191/ using document root <none> and password file <none>
I20260808 10:05:30.403622 13822 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.403680 13822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.405048 13822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.405704 13843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.405879 13822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.405987 13822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "1de46d62b4924a8d95dbfc143e5f8204"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "5e1d507b057cb5cf313a7fe21621f13a"
server_key_iv: "e935378e92f4bafa4dc889bf615a87d8"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.406386 13822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.435640 13822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.437561 13822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.437731 13822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.437994 13822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:30.438311 13822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:30.438372 13822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.438416 13822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:30.438464 13822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.444407 13822 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33239
I20260808 10:05:30.444451 14008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33239 every 8 connection(s)
I20260808 10:05:30.444842 13822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:30.451182 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 13822
I20260808 10:05:30.454577 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.445602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:35041 (local address 127.24.162.190:41203)
0808 10:05:30.445688 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.445690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.445865 (+ 175us) server_negotiation.cc:407] Connection header received
0808 10:05:30.446321 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.446322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.446334 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.446385 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:30.447363 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.448060 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.449142 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.449335 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.449393 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.451589 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:30.451591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:30.451591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:30.451598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:30.453223 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.453612 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.453614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.453614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.453672 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.453937 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.453938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.453939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.454054 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:30.454130 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.454407 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.454486 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:05:30.454752 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.445560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:35041)
0808 10:05:30.445743 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.445778 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.446244 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.446443 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.446447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.447106 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:30.447290 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.447295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.448151 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.448152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.449069 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.449071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.449408 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.449409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.449499 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.450033 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:30.450041 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:30.451502 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:30.453305 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.453312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.453315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.453535 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.453746 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:30.453747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:30.453747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:30.453859 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:30.454122 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:30.454128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:30.454267 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.454371 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.454522 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260808 10:05:30.455257 14011 heartbeater.cc:347] Connected to a master server at 127.24.162.190:41203
I20260808 10:05:30.455355 14011 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:30.455549 14011 heartbeater.cc:511] Master 127.24.162.190:41203 requested a full tablet report, sending...
I20260808 10:05:30.455928 13369 ts_manager.cc:194] Registered new tserver with Master: 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239)
I20260808 10:05:30.456245 13369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:35041
I20260808 10:05:30.458451 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.451597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.24.162.190:41203)
0808 10:05:30.451735 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.451737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.451895 (+ 158us) server_negotiation.cc:407] Connection header received
0808 10:05:30.451973 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.451974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.451987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.452073 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:30.452650 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.453277 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.453990 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.454205 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.454290 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.455690 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:30.455691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:30.455692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:30.455699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:30.457219 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.457600 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.457601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.457602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.457663 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:30.457924 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:30.457925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:30.457925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:30.458045 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:30.458235 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.458271 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.458347 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":93,"threads_started":1}
I20260808 10:05:30.459190 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:05:30.461921 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.459509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.24.162.190:41203)
0808 10:05:30.459558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.459560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.459678 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:05:30.459775 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.459776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.459814 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.459888 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:05:30.459978 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.460420 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.461147 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.461351 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.461436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.461598 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:05:30.461690 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:05:30.461779 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.461829 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.461849 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:05:30.462179 13369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:05:30.463855 13369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:05:30.466089 13537 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e (DEFAULT_TABLE table=test-table [id=c53981c499e44d5d8d5fe57ea32b0fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:30.466327 13537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:30.467944 13485 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Bootstrap starting.
I20260808 10:05:30.468261 13485 tablet_bootstrap.cc:654] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:30.468861 13485 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: No bootstrap required, opened a new log
I20260808 10:05:30.468912 13485 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:30.469036 13485 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.469101 13485 raft_consensus.cc:374] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:30.469144 13485 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.469197 13485 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.469288 13485 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.469532 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.465781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33239 (local address 127.0.0.1:46186)
0808 10:05:30.466063 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.466075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.466119 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.466400 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.466401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.466403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:30.466438 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.466440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.467429 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.467431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.468267 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.468269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.469334 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.469335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.469392 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.469396 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.469430 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.469474 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264}
I20260808 10:05:30.469941 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.465843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.24.162.131:33239)
0808 10:05:30.466261 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.466266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.466276 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:05:30.466314 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.466317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.466333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.466405 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:30.466490 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.467333 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.468363 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.469232 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.469708 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.469774 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.469800 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.469845 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":111,"threads_started":1}
I20260808 10:05:30.470294 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.465805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40369 (local address 127.0.0.1:47834)
0808 10:05:30.466210 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.466219 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.466291 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.466600 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.466601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.466603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:30.466633 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.466635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.467918 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.467919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.468946 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.468949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.469857 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.469858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.469936 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.469941 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.470183 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.470227 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":99,"threads_started":1}
I20260808 10:05:30.470333 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.466058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.24.162.130:40369)
0808 10:05:30.466347 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.466352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.466365 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:05:30.466473 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.466475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.466493 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.466656 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:30.466733 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.467805 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.469028 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.469765 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.469848 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.469900 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.470210 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.470252 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
I20260808 10:05:30.470597 13907 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e (DEFAULT_TABLE table=test-table [id=c53981c499e44d5d8d5fe57ea32b0fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:30.470994 13907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:30.471131 13726 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e (DEFAULT_TABLE table=test-table [id=c53981c499e44d5d8d5fe57ea32b0fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:30.471403 13726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:30.473701 14021 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Bootstrap starting.
I20260808 10:05:30.473870 14023 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Bootstrap starting.
I20260808 10:05:30.474426 14021 tablet_bootstrap.cc:654] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:30.474498 14023 tablet_bootstrap.cc:654] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:30.474905 14021 log.cc:824] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.475070 14023 log.cc:824] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.475721 14021 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: No bootstrap required, opened a new log
I20260808 10:05:30.475776 14021 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:05:30.475807 14023 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: No bootstrap required, opened a new log
I20260808 10:05:30.475874 14023 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:30.477303 14021 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.477432 14021 raft_consensus.cc:374] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:30.477454 14021 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de46d62b4924a8d95dbfc143e5f8204, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.477402 14023 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.477584 14023 raft_consensus.cc:374] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:30.477576 14021 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.477625 14023 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbe8cb5eeb143ba8403b1821611abc1, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.477756 14023 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.477871 14011 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
I20260808 10:05:30.478061 14023 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:30.478083 13816 heartbeater.cc:503] Master 127.24.162.190:41203 was elected leader, sending a full tablet report...
I20260808 10:05:30.477873 14021 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 10:05:30.552459 13817 tablet.cc:2369] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:05:30.633571 14028 raft_consensus.cc:482] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:30.633708 14028 raft_consensus.cc:504] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.634150 14028 leader_election.cc:288] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795)
I20260808 10:05:30.637204 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.634477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:36549 (local address 127.24.162.131:33239)
0808 10:05:30.634528 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.634530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.634541 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:05:30.634575 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.634576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.634586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.634657 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:30.634862 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.635404 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636420 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.636812 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636878 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.636912 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.637079 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.637122 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:05:30.637224 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.634484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:58191 (local address 127.24.162.129:41795)
0808 10:05:30.634531 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.634533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.634663 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:05:30.634756 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.634757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.634767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.634823 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:30.635009 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.635490 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636459 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.636826 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636918 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.636926 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.637108 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.637156 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:05:30.637224 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.634371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33239 (local address 127.24.162.130:36549)
0808 10:05:30.634435 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.634446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.634491 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.634680 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.634681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.634683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:30.634765 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.634767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.635491 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.635492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.636318 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.636320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.636908 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.636991 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.636995 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.637047 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.637112 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":29}
I20260808 10:05:30.637238 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.634388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.24.162.130:58191)
0808 10:05:30.634581 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.634592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.634650 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:30.634851 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:30.634852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:30.634853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:30.634943 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.634945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.635573 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.635573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.636331 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:30.636332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.636940 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.636941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:30.637003 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.637007 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:05:30.637079 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:05:30.637116 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260808 10:05:30.637633 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de46d62b4924a8d95dbfc143e5f8204" is_pre_election: true
I20260808 10:05:30.637836 13942 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbe8cb5eeb143ba8403b1821611abc1 in term 0.
I20260808 10:05:30.637861 13570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e44231bdd547a58131a66be62f5e0a" is_pre_election: true
I20260808 10:05:30.638020 13570 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbe8cb5eeb143ba8403b1821611abc1 in term 0.
I20260808 10:05:30.638096 13661 leader_election.cc:302] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de46d62b4924a8d95dbfc143e5f8204, 2dbe8cb5eeb143ba8403b1821611abc1; no voters:
I20260808 10:05:30.638259 14028 raft_consensus.cc:2781] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:05:30.638335 14028 raft_consensus.cc:482] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:30.638374 14028 raft_consensus.cc:3037] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:30.639158 14028 raft_consensus.cc:504] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.639325 14028 leader_election.cc:288] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [CANDIDATE]: Term 1 election: Requested vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795)
I20260808 10:05:30.639480 13942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de46d62b4924a8d95dbfc143e5f8204"
I20260808 10:05:30.639506 13570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e44231bdd547a58131a66be62f5e0a"
I20260808 10:05:30.639565 13942 raft_consensus.cc:3037] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:30.639623 13570 raft_consensus.cc:3037] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:30.640234 13942 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbe8cb5eeb143ba8403b1821611abc1 in term 1.
I20260808 10:05:30.640405 13661 leader_election.cc:302] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de46d62b4924a8d95dbfc143e5f8204, 2dbe8cb5eeb143ba8403b1821611abc1; no voters:
I20260808 10:05:30.640515 14028 raft_consensus.cc:2781] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:30.640676 14028 raft_consensus.cc:686] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 LEADER]: Becoming Leader. State: Replica: 2dbe8cb5eeb143ba8403b1821611abc1, State: Running, Role: LEADER
I20260808 10:05:30.640795 14028 consensus_queue.cc:237] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.641021 13570 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbe8cb5eeb143ba8403b1821611abc1 in term 1.
I20260808 10:05:30.641628 13369 catalog_manager.cc:5495] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } health_report { overall_health: UNKNOWN } } }
I20260808 10:05:30.655774 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.653425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.24.162.130:40369)
0808 10:05:30.653496 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:30.653499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:30.653510 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:05:30.653577 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:30.653578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:30.653594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:30.653658 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:05:30.653808 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.654264 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.655120 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:30.655302 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:30.655357 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:30.655493 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:05:30.655584 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:05:30.655640 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:30.655685 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 10:05:30.655701 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:05:30.657641 13570 raft_consensus.cc:1261] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Refusing update from remote peer 2dbe8cb5eeb143ba8403b1821611abc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:30.657641 13942 raft_consensus.cc:1261] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Refusing update from remote peer 2dbe8cb5eeb143ba8403b1821611abc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:30.657994 14031 consensus_queue.cc:1035] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:30.658141 14028 consensus_queue.cc:1035] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:30.661496 14042 mvcc.cc:205] Tried to move back new op lower bound from 7316207741570879488 to 7316207741505228800. Current Snapshot: MvccSnapshot[applied={T|T < 7316207741570879488 or (T in {7316207741570879488})}]
I20260808 10:05:30.662999 13367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:30.663110 13367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:30.663847 13367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Sending DeleteTablet for 3 replicas of tablet 2d3e19e9c3d948c6bcf6787f0c1b126e
I20260808 10:05:30.664206 13907 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:30 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 10:05:30.664208 13726 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:30 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:47834
I20260808 10:05:30.664258 13537 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:30 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:37836
I20260808 10:05:30.664309 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 13425
I20260808 10:05:30.664407 13641 tablet_replica.cc:330] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: stopping tablet replica
I20260808 10:05:30.664461 13641 raft_consensus.cc:2229] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:30.664516 13641 raft_consensus.cc:2258] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:30.664744 14052 tablet_replica.cc:330] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: stopping tablet replica
I20260808 10:05:30.664901 14052 raft_consensus.cc:2229] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:30.665086 14052 raft_consensus.cc:2258] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:30.666800 14052 ts_tablet_manager.cc:1905] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:30.669106 14052 ts_tablet_manager.cc:1918] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:30.669199 14052 log.cc:1196] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/2d3e19e9c3d948c6bcf6787f0c1b126e
I20260808 10:05:30.669487 14052 ts_tablet_manager.cc:1939] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Deleting consensus metadata
I20260808 10:05:30.670082 13342 catalog_manager.cc:4841] TS 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239): tablet 2d3e19e9c3d948c6bcf6787f0c1b126e (table test-table [id=c53981c499e44d5d8d5fe57ea32b0fd2]) successfully deleted
I20260808 10:05:30.680313 14056 tablet_replica.cc:330] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: stopping tablet replica
I20260808 10:05:30.680452 14056 raft_consensus.cc:2229] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 LEADER]: Raft consensus shutting down.
W20260808 10:05:30.680451 13345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:41795 (error 108)
I20260808 10:05:30.680563 14056 raft_consensus.cc:2258] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:30.681036 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.680889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.0.0.1:37844)
0808 10:05:30.680941 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.680999 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 10:05:30.681216 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 13637
W20260808 10:05:30.681172 13345 catalog_manager.cc:4584] TS 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41795: connect: Connection refused (error 111)
I20260808 10:05:30.688512 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:30.688372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40369 (local address 127.0.0.1:47850)
0808 10:05:30.688438 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.688472 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.130:40369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 10:05:30.688539 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 13822
I20260808 10:05:30.708228 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 13316
I20260808 10:05:30.718802 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--webserver_interface=127.24.162.190
--webserver_port=46341
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:41203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:30.805850 14059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:30.806077 14059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:30.806121 14059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:30.807366 14059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:30.807428 14059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:30.807456 14059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:30.807490 14059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:30.807513 14059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:30.808985 14059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:41203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:30.809257 14059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:30.809471 14059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.814369 14066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.814348 14065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.814340 14069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.814528 14059 server_base.cc:1034] running on GCE node
I20260808 10:05:30.814819 14059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.815162 14059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.816321 14059 hybrid_clock.cc:648] HybridClock initialized: now 1786183530816309 us; error 36 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.818975 14059 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:30.819469 14059 webserver.cc:460] Webserver started at http://127.24.162.190:46341/ using document root <none> and password file <none>
I20260808 10:05:30.819725 14059 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.819772 14059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.821142 14059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.821864 14079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.822041 14059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.822134 14059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1f03cb0a996f4060bf678550400a4002"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "1c3c7d5c805aa6de6c3a3f5e5f5f674d"
server_key_iv: "80125bac7d70f8649ca03596478be510"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.822492 14059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.835521 14059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.837368 14059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.837543 14059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.841894 14059 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:41203
I20260808 10:05:30.841928 14145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:41203 every 8 connection(s)
I20260808 10:05:30.842304 14059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:30.843839 14146 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:05:30.844384 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 14059
I20260808 10:05:30.844604 14146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap starting.
I20260808 10:05:30.844671 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:41795
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=41799
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:30.853654 14146 log.cc:824] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.856012 14146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:30.856341 14146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002: Bootstrap complete.
I20260808 10:05:30.858678 14146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.858954 14146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.859179 14146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.859283 14146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:30.859345 14146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:30.859398 14146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:05:30.860435 14146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.860560 14146 leader_election.cc:302] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f03cb0a996f4060bf678550400a4002; no voters:
I20260808 10:05:30.860791 14146 leader_election.cc:288] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:05:30.860843 14150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:05:30.861011 14150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [term 3 LEADER]: Becoming Leader. State: Replica: 1f03cb0a996f4060bf678550400a4002, State: Running, Role: LEADER
I20260808 10:05:30.861091 14146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:30.861171 14150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } }
I20260808 10:05:30.861452 14151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:30.861564 14151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:30.861445 14150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f03cb0a996f4060bf678550400a4002. Latest consensus state: current_term: 3 leader_uuid: "1f03cb0a996f4060bf678550400a4002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f03cb0a996f4060bf678550400a4002" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 41203 } } }
I20260808 10:05:30.861775 14150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f03cb0a996f4060bf678550400a4002 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:30.862015 14159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:30.862886 14159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:30.863027 14159 catalog_manager.cc:1259] Loaded cluster ID: b6011d99454748ed8f59dc50075acfb5
I20260808 10:05:30.863054 14159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 10:05:30.863628 14159 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa054adf9ad google::LogMessage::Fail() at ??:0
@ 0x7fa054ae288a google::LogMessage::SendToLog() at ??:0
@ 0x7fa054adf4de google::LogMessage::Flush() at ??:0
@ 0x7fa054ae0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa05ca040f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa05ca0d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa0569ad0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa0569a2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa0578066db start_thread at ??:0
@ 0x7fa0535dd61f clone at ??:0
W20260808 10:05:30.945919 14148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:30.946111 14148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:30.946133 14148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:30.947368 14148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:30.947409 14148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:30.947436 14148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:30.948895 14148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:41795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=41799
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:30.949144 14148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:30.949366 14148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:30.954559 14202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:30.954576 14198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.954811 14148 server_base.cc:1034] running on GCE node
W20260808 10:05:30.954669 14199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:30.955153 14148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:30.955428 14148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:30.956588 14148 hybrid_clock.cc:648] HybridClock initialized: now 1786183530956576 us; error 28 us; skew 500 ppm
Aug 08 10:05:30 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:30.959257 14148 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:30.959750 14148 webserver.cc:460] Webserver started at http://127.24.162.129:41799/ using document root <none> and password file <none>
I20260808 10:05:30.960001 14148 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:30.960071 14148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:30.961448 14148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:05:30.962164 14213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:30.962358 14148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:30.962428 14148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "69e44231bdd547a58131a66be62f5e0a"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "bc96c77c026b77ae2605c0acc8c92f53"
server_key_iv: "1ba76399d66d237eb1887a7ea00f049b"
server_key_version: "encryptionkey@0"
I20260808 10:05:30.962785 14148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:30.971560 14148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:30.973672 14148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:30.973826 14148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:30.974066 14148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:30.974614 14223 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:05:30.976538 14148 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:05:30.976600 14148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:05:30.976645 14148 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:05:30.977265 14148 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:05:30.977327 14148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:30.977396 14223 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Bootstrap starting.
I20260808 10:05:30.984694 14148 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41795
I20260808 10:05:30.985067 14148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:30.985651 14381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41795 every 8 connection(s)
I20260808 10:05:30.988586 14223 log.cc:824] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:30.990639 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 14148
I20260808 10:05:30.990958 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:40369
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=33203
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:30.991266 14223 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:30.991604 14223 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Bootstrap complete.
I20260808 10:05:30.992889 14223 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260808 10:05:30.994283 14223 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.994611 14223 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Initialized, Role: FOLLOWER
I20260808 10:05:30.996214 14223 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:30.996531 14223 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
W20260808 10:05:31.069344 14384 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:05:30.985729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:44227)
0808 10:05:30.985915 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:30.985957 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:30.986407 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:31.069110 (+ 82703us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260808 10:05:31.069639 14382 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:41203 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:41203: Client connection negotiation failed: client connection to 127.24.162.190:41203: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:05:31.070034 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.069917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:33485)
0808 10:05:31.069971 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.070013 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:05:31.070415 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.070313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:57521)
0808 10:05:31.070364 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.070391 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:05:31.070533 14382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:05:31.092487 14386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:31.092700 14386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:31.092734 14386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:31.093890 14386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:31.093956 14386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:31.094003 14386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:31.095609 14386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:40369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=33203
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:31.095894 14386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:31.096163 14386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:31.101377 14400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:31.101377 14399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:31.101575 14386 server_base.cc:1034] running on GCE node
W20260808 10:05:31.101455 14404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:31.101882 14386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:31.102151 14386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:31.103309 14386 hybrid_clock.cc:648] HybridClock initialized: now 1786183531103281 us; error 44 us; skew 500 ppm
Aug 08 10:05:31 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:31.106000 14386 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:31.106463 14386 webserver.cc:460] Webserver started at http://127.24.162.130:33203/ using document root <none> and password file <none>
I20260808 10:05:31.106745 14386 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:31.106827 14386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:31.108399 14386 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:31.109156 14416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:31.109421 14386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:05:31.109519 14386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "160ee425755422d814392520643e3326"
server_key_iv: "7978effb11bc373cd5ae71619d7fbe4b"
server_key_version: "encryptionkey@0"
I20260808 10:05:31.109859 14386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:31.118793 14386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:31.121012 14386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:31.121271 14386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:31.121497 14386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:31.121956 14424 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:05:31.123420 14386 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:05:31.123485 14386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:05:31.123522 14386 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:05:31.124172 14386 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:05:31.124228 14386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:05:31.124333 14424 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Bootstrap starting.
I20260808 10:05:31.131047 14386 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40369
I20260808 10:05:31.131150 14572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40369 every 8 connection(s)
I20260808 10:05:31.131458 14386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:31.132705 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.132150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:50795)
0808 10:05:31.132364 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.132475 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
W20260808 10:05:31.132942 14573 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:41203 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:41203: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
I20260808 10:05:31.133283 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.133177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:36193)
0808 10:05:31.133232 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.133262 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:05:31.133625 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.133541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:38615)
0808 10:05:31.133586 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.133609 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:05:31.133755 14573 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:05:31.136935 14424 log.cc:824] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:31.138867 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 14386
I20260808 10:05:31.139213 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:33239
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:41203
--webserver_port=42191
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:05:31.139947 14424 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:31.140221 14424 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Bootstrap complete.
I20260808 10:05:31.141589 14424 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.011s
I20260808 10:05:31.143251 14424 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:31.143532 14424 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbe8cb5eeb143ba8403b1821611abc1, State: Initialized, Role: FOLLOWER
I20260808 10:05:31.143714 14424 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:31.144022 14424 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260808 10:05:31.144356 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.144211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:34263)
0808 10:05:31.144266 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.144328 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:05:31.144711 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.144595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:37193)
0808 10:05:31.144653 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.144680 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:05:31.224820 14577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:31.225060 14577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:31.225101 14577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:31.226321 14577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:31.226384 14577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:31.226431 14577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:31.228010 14577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:33239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=42191
--tserver_master_addrs=127.24.162.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:31.228286 14577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:31.228520 14577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:31.233960 14589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:31.233970 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:31.233942 14586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:31.234556 14577 server_base.cc:1034] running on GCE node
I20260808 10:05:31.234752 14577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:31.235034 14577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:31.236182 14577 hybrid_clock.cc:648] HybridClock initialized: now 1786183531236172 us; error 31 us; skew 500 ppm
Aug 08 10:05:31 dist-test-slave-nfb5 krb5kdc[12542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:31.238739 14577 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:31.239248 14577 webserver.cc:460] Webserver started at http://127.24.162.131:42191/ using document root <none> and password file <none>
I20260808 10:05:31.239526 14577 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:31.239598 14577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:31.241024 14577 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:05:31.241627 14598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:31.241842 14577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:05:31.241942 14577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "1de46d62b4924a8d95dbfc143e5f8204"
format_stamp: "Formatted at 2026-08-08 10:05:29 on dist-test-slave-nfb5"
server_key: "5e1d507b057cb5cf313a7fe21621f13a"
server_key_iv: "e935378e92f4bafa4dc889bf615a87d8"
server_key_version: "encryptionkey@0"
I20260808 10:05:31.242297 14577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:31.260735 14577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:31.262542 14577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:31.262717 14577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:31.262979 14577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:31.263296 14577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:31.263352 14577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:31.263396 14577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:31.263422 14577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:31.269436 14577 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33239
I20260808 10:05:31.269490 14757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33239 every 8 connection(s)
I20260808 10:05:31.269834 14577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:05:31.270905 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.270511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:50741)
0808 10:05:31.270673 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.270766 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
W20260808 10:05:31.271339 14758 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:41203 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:41203: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
I20260808 10:05:31.271638 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.271549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:43049)
0808 10:05:31.271595 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.271622 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 10:05:31.271946 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:31.271867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:50313)
0808 10:05:31.271908 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:31.271931 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:05:31.272080 14758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:05:31.276593 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 14577
I20260808 10:05:32.071203 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.070895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:54359)
0808 10:05:32.071074 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.071114 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
I20260808 10:05:32.145468 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.145231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:47517)
0808 10:05:32.145372 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.145410 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260808 10:05:32.272614 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.272400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:58483)
0808 10:05:32.272525 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.272560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260808 10:05:32.380897 14769 raft_consensus.cc:482] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:32.381070 14769 raft_consensus.cc:504] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:32.381421 14769 leader_election.cc:288] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
Aug 08 10:05:32 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.131@KRBTEST.COM
Aug 08 10:05:32 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.130@KRBTEST.COM
I20260808 10:05:32.391798 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.381579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40369 (local address 127.24.162.129:45091)
0808 10:05:32.381636 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.381646 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:32.381705 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:32.382277 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:32.382280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:32.382725 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:32.383006 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:32.383013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.384092 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.384093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:32.384899 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:32.384901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.385489 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.385490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:32.385611 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:32.386199 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:32.386207 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:32.388304 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:32.390232 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:32.390234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:32.390463 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:32.390732 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:32.390734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:32.390849 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:32.391259 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:32.391266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:32.391402 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:05:32.391512 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:05:32.391702 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":5}
I20260808 10:05:32.391798 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.381720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:45091 (local address 127.24.162.130:40369)
0808 10:05:32.381890 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:32.381898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:32.381917 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:05:32.382036 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:32.382038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:32.382078 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:32.382195 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:32.383131 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.383983 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.384995 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.385393 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.385505 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:32.388427 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:32.388431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:32.388431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:32.388440 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:32.390130 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390546 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:32.390549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:32.390550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:32.390658 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390962 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:32.390963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:32.390964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:32.391183 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:32.391279 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:32.391564 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 10:05:32.391691 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
I20260808 10:05:32.391856 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.381716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:53889 (local address 127.24.162.131:33239)
0808 10:05:32.381876 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:32.381886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:32.381916 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:05:32.382480 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:32.382483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:32.382532 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:32.382674 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:32.383129 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.383996 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.385006 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.385402 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.385525 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:32.387990 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:32.387994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:32.387994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:32.388005 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:32.390070 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390539 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:32.390543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:32.390543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:32.390648 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390938 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:32.390939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:32.390939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:32.391183 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:32.391279 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:32.391570 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 10:05:32.391725 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260808 10:05:32.391920 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.381579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33239 (local address 127.24.162.129:53889)
0808 10:05:32.381690 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.381699 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:32.381737 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:32.382666 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:32.382667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:32.382850 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:32.383006 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:32.383013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.384120 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.384121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:32.384898 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:32.384900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:32.385503 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:32.385504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:32.385609 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:32.386246 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:32.386250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:32.387886 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:32.390158 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:32.390168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:32.390452 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:32.390721 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:32.390722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:32.390722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:32.390845 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:32.391262 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:32.391269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:32.391402 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 10:05:32.391512 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:05:32.391787 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":208,"thread_start_us":48,"threads_started":1}
I20260808 10:05:32.393116 14508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" is_pre_election: true
W20260808 10:05:32.393210 14217 leader_election.cc:341] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239): Not found: Tablet not found: 2d3e19e9c3d948c6bcf6787f0c1b126e
I20260808 10:05:32.393286 14508 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 1.
I20260808 10:05:32.393554 14220 leader_election.cc:302] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2dbe8cb5eeb143ba8403b1821611abc1, 69e44231bdd547a58131a66be62f5e0a; no voters: 1de46d62b4924a8d95dbfc143e5f8204
I20260808 10:05:32.393702 14769 raft_consensus.cc:2781] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:05:32.393779 14769 raft_consensus.cc:482] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:32.393814 14769 raft_consensus.cc:3037] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:05:32.394994 14769 raft_consensus.cc:504] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:32.395161 14769 leader_election.cc:288] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 election: Requested vote from peers 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239), 2dbe8cb5eeb143ba8403b1821611abc1 (127.24.162.130:40369)
I20260808 10:05:32.395354 14508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3e19e9c3d948c6bcf6787f0c1b126e" candidate_uuid: "69e44231bdd547a58131a66be62f5e0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2dbe8cb5eeb143ba8403b1821611abc1"
I20260808 10:05:32.395433 14508 raft_consensus.cc:3037] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 1 FOLLOWER]: Advancing to term 2
W20260808 10:05:32.395422 14217 leader_election.cc:341] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239): Not found: Tablet not found: 2d3e19e9c3d948c6bcf6787f0c1b126e
I20260808 10:05:32.396468 14508 raft_consensus.cc:2454] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69e44231bdd547a58131a66be62f5e0a in term 2.
I20260808 10:05:32.396708 14220 leader_election.cc:302] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2dbe8cb5eeb143ba8403b1821611abc1, 69e44231bdd547a58131a66be62f5e0a; no voters: 1de46d62b4924a8d95dbfc143e5f8204
I20260808 10:05:32.396806 14769 raft_consensus.cc:2781] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:05:32.396975 14769 raft_consensus.cc:686] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [term 2 LEADER]: Becoming Leader. State: Replica: 69e44231bdd547a58131a66be62f5e0a, State: Running, Role: LEADER
I20260808 10:05:32.397076 14769 consensus_queue.cc:237] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:32.397372 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.397256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:39523)
0808 10:05:32.397293 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.397336 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 10:05:32.773139 14508 raft_consensus.cc:1261] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 2dbe8cb5eeb143ba8403b1821611abc1 [term 2 FOLLOWER]: Refusing update from remote peer 69e44231bdd547a58131a66be62f5e0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260808 10:05:32.773563 14769 consensus_queue.cc:1035] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260808 10:05:32.774529 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.774341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:33613)
0808 10:05:32.774431 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.774480 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":57,"threads_started":1}
I20260808 10:05:32.775478 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.775353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:35063)
0808 10:05:32.775412 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.775445 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:05:32.775513 14217 consensus_peers.cc:487] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a -> Peer 1de46d62b4924a8d95dbfc143e5f8204 (127.24.162.131:33239): Couldn't send request to peer 1de46d62b4924a8d95dbfc143e5f8204. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d3e19e9c3d948c6bcf6787f0c1b126e. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:05:32.775763 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:32.775675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:51839)
0808 10:05:32.775715 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:32.775741 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 10:05:33.198745 14786 ts_tablet_manager.cc:927] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Initiating tablet copy from peer 69e44231bdd547a58131a66be62f5e0a (127.24.162.129:41795)
I20260808 10:05:33.198997 14786 tablet_copy_client.cc:316] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: tablet copy: Beginning tablet copy session from remote peer at address 127.24.162.129:41795
Aug 08 10:05:33 dist-test-slave-nfb5 krb5kdc[12542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:05:33.207515 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.199378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:45305 (local address 127.24.162.129:41795)
0808 10:05:33.199513 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:33.199519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:33.199547 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:05:33.199624 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:33.199627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:33.199644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:33.199717 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:33.200227 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:33.201070 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:33.201938 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:33.202262 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:33.202359 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:33.204461 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:33.204470 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:33.204471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:33.204478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:33.206086 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:33.206515 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:33.206518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:33.206518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:33.206585 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:33.206823 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:33.206825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:33.206825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:33.207042 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:33.207138 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:33.207323 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 10:05:33.207410 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":48,"threads_started":1}
I20260808 10:05:33.207527 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.199278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41795 (local address 127.24.162.131:45305)
0808 10:05:33.199468 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.199488 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:33.199570 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:33.199722 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:33.199728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:33.200066 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:33.200134 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:33.200137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:33.201170 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:33.201171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:33.201843 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:33.201845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:33.202361 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:33.202362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:33.202462 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:33.202990 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:33.202995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:33.204366 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:33.206202 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:33.206216 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:33.206218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:33.206425 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:33.206668 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:33.206669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:33.206670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:33.206745 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:33.207116 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:33.207124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:33.207179 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:05:33.207286 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:05:33.207409 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260808 10:05:33.208029 14343 tablet_copy_service.cc:140] P 69e44231bdd547a58131a66be62f5e0a: Received BeginTabletCopySession request for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e from peer 1de46d62b4924a8d95dbfc143e5f8204 ({username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:45305)
I20260808 10:05:33.208091 14343 tablet_copy_service.cc:161] P 69e44231bdd547a58131a66be62f5e0a: Beginning new tablet copy session on tablet 2d3e19e9c3d948c6bcf6787f0c1b126e from peer 1de46d62b4924a8d95dbfc143e5f8204 at {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:45305: session id = 1de46d62b4924a8d95dbfc143e5f8204-2d3e19e9c3d948c6bcf6787f0c1b126e
I20260808 10:05:33.208527 14343 tablet_copy_source_session.cc:205] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 69e44231bdd547a58131a66be62f5e0a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:05:33.209566 14786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3e19e9c3d948c6bcf6787f0c1b126e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:33.212241 14786 tablet_copy_client.cc:791] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: tablet copy: Starting download of 0 data blocks...
I20260808 10:05:33.212409 14786 tablet_copy_client.cc:655] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: tablet copy: Starting download of 1 WAL segments...
I20260808 10:05:33.213251 14786 tablet_copy_client.cc:531] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:05:33.213967 14786 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Bootstrap starting.
I20260808 10:05:33.215438 14786 log.cc:824] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:33.217804 14786 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:05:33.218044 14786 tablet_bootstrap.cc:492] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Bootstrap complete.
I20260808 10:05:33.218183 14786 ts_tablet_manager.cc:1397] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 10:05:33.219444 14786 raft_consensus.cc:348] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:33.219648 14786 raft_consensus.cc:729] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de46d62b4924a8d95dbfc143e5f8204, State: Initialized, Role: FOLLOWER
I20260808 10:05:33.219825 14786 consensus_queue.cc:260] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de46d62b4924a8d95dbfc143e5f8204" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33239 } } peers { permanent_uuid: "2dbe8cb5eeb143ba8403b1821611abc1" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40369 } } peers { permanent_uuid: "69e44231bdd547a58131a66be62f5e0a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41795 } }
I20260808 10:05:33.220118 14786 ts_tablet_manager.cc:1428] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260808 10:05:33.220393 14343 tablet_copy_service.cc:342] P 69e44231bdd547a58131a66be62f5e0a: Request end of tablet copy session 1de46d62b4924a8d95dbfc143e5f8204-2d3e19e9c3d948c6bcf6787f0c1b126e received from {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:45305
I20260808 10:05:33.220414 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.220326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:34555)
0808 10:05:33.220361 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.220388 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 10:05:33.220474 14343 tablet_copy_service.cc:433] P 69e44231bdd547a58131a66be62f5e0a: ending tablet copy session 1de46d62b4924a8d95dbfc143e5f8204-2d3e19e9c3d948c6bcf6787f0c1b126e on tablet 2d3e19e9c3d948c6bcf6787f0c1b126e with peer 1de46d62b4924a8d95dbfc143e5f8204
I20260808 10:05:33.220963 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.220862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:48683)
0808 10:05:33.220911 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.220940 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:05:33.578649 14693 raft_consensus.cc:1203] T 2d3e19e9c3d948c6bcf6787f0c1b126e P 1de46d62b4924a8d95dbfc143e5f8204 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260808 10:05:33.578713 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.578470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:42613)
0808 10:05:33.578606 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.578653 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260808 10:05:33.579161 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.579017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:51157)
0808 10:05:33.579082 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.579121 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:05:33.776345 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:33.776152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:37427)
0808 10:05:33.776258 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:33.776296 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260808 10:05:34.579535 14802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:34.579253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:41213)
0808 10:05:34.579403 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:34.579441 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 10:05:34.579957 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:34.579635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:59319)
0808 10:05:34.579847 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:34.579891 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":113,"threads_started":1}
I20260808 10:05:34.777102 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:34.776886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:40845)
0808 10:05:34.777016 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:34.777055 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260808 10:05:35.580322 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:35.580050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:35471)
0808 10:05:35.580220 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:35.580252 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260808 10:05:35.580672 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:35.580439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:56739)
0808 10:05:35.580597 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:35.580623 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260808 10:05:35.777908 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:35.777671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:49477)
0808 10:05:35.777815 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:35.777859 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260808 10:05:36.581077 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:36.580839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:46283)
0808 10:05:36.580993 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:36.581028 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":134,"threads_started":1}
I20260808 10:05:36.581461 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:36.581218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:40161)
0808 10:05:36.581350 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:36.581404 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260808 10:05:36.581501 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:36.581394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:47487)
0808 10:05:36.581445 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:36.581477 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:05:36.778620 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:36.778408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:37819)
0808 10:05:36.778535 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:36.778573 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260808 10:05:37.582182 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:37.581925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:36501)
0808 10:05:37.582081 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:37.582126 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260808 10:05:37.582182 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:37.581923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:38695)
0808 10:05:37.582093 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:37.582126 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260808 10:05:37.779436 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:37.779193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:33869)
0808 10:05:37.779343 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:37.779388 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260808 10:05:38.583012 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:38.582712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:41637)
0808 10:05:38.582872 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:38.582955 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":116,"threads_started":1}
I20260808 10:05:38.583012 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:38.582736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:40061)
0808 10:05:38.582876 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:38.582956 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 10:05:38.780218 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:38.779965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:38391)
0808 10:05:38.780115 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:38.780164 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260808 10:05:39.583765 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:39.583540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:56175)
0808 10:05:39.583670 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:39.583706 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 10:05:39.583943 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:39.583568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:51963)
0808 10:05:39.583828 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:39.583890 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":218,"threads_started":1}
I20260808 10:05:39.780956 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:39.780715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:60425)
0808 10:05:39.780866 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:39.780909 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260808 10:05:40.584565 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:40.584329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:54079)
0808 10:05:40.584485 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:40.584518 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260808 10:05:40.584628 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:40.584418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:50849)
0808 10:05:40.584556 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:40.584588 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260808 10:05:40.781865 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:40.781598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:35155)
0808 10:05:40.781751 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:40.781800 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260808 10:05:41.585342 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:41.585102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:56711)
0808 10:05:41.585249 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:41.585293 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260808 10:05:41.585361 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:41.585104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:46563)
0808 10:05:41.585281 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:41.585318 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260808 10:05:41.782668 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:41.782420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:56937)
0808 10:05:41.782580 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:41.782620 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260808 10:05:42.586092 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:42.585860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:33223)
0808 10:05:42.586009 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:42.586044 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260808 10:05:42.586092 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:42.585831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:48451)
0808 10:05:42.586013 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:42.586051 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":127,"threads_started":1}
I20260808 10:05:42.783454 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:42.783221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:52517)
0808 10:05:42.783359 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:42.783404 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260808 10:05:43.586866 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:43.586628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:47107)
0808 10:05:43.586775 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:43.586816 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 10:05:43.586884 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:43.586628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:51673)
0808 10:05:43.586809 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:43.586842 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260808 10:05:43.784238 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:43.783989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:52333)
0808 10:05:43.784127 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:43.784186 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 10:05:44.587759 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:44.587446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:37567)
0808 10:05:44.587618 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:44.587691 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260808 10:05:44.587774 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:44.587509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:35463)
0808 10:05:44.587690 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:44.587730 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260808 10:05:44.784960 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:44.784715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:53729)
0808 10:05:44.784870 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:44.784911 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260808 10:05:45.588400 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:45.588182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.131:48503)
0808 10:05:45.588288 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:45.588321 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260808 10:05:45.588500 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:45.588219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.129:60653)
0808 10:05:45.588393 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:45.588429 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260808 10:05:45.785759 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:45.785519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:41203 (local address 127.24.162.130:55893)
0808 10:05:45.785667 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:45.785711 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260808 10:05:46.279946 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 14148
I20260808 10:05:46.291708 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 14386
I20260808 10:05:46.302217 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 14577
2026-08-08T10:05:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18679 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:05:46 dist-test-slave-nfb5 krb5kdc[14855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:05:46 dist-test-slave-nfb5 krb5kdc[14855](info): set up 2 sockets
Aug 08 10:05:46 dist-test-slave-nfb5 krb5kdc[14855](info): commencing operation
krb5kdc: starting...
W20260808 10:05:47.524681 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:05:47 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:05:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:05:47.653455 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:34941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:47.740916 14874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:47.741127 14874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:47.741149 14874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:47.742313 14874 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:05:47.742354 14874 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:05:47.742368 14874 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:05:47.742380 14874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:05:47.742394 14874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:47.742408 14874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:47.742460 14874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:05:47.742477 14874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:05:47.744058 14874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:34941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:47.744338 14874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:47.744590 14874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:47.749554 14885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:47.749562 14888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:47.749571 14886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:47.749822 14874 server_base.cc:1034] running on GCE node
I20260808 10:05:47.750299 14874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:47.750592 14874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:47.751741 14874 hybrid_clock.cc:648] HybridClock initialized: now 1786183547751724 us; error 34 us; skew 500 ppm
Aug 08 10:05:47 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183547, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:47.754340 14874 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:05:47.754818 14874 webserver.cc:460] Webserver started at http://127.24.162.190:35281/ using document root <none> and password file <none>
I20260808 10:05:47.755115 14874 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:47.755191 14874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:47.755332 14874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:47.756369 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "1022331640fa4a76b6d0e8bcee763466"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "6748f9a4cbb65cae6c497f6f9e3e0769"
server_key_iv: "8ea49be1130760b2f940ed6ec59ea564"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.756733 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "1022331640fa4a76b6d0e8bcee763466"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "6748f9a4cbb65cae6c497f6f9e3e0769"
server_key_iv: "8ea49be1130760b2f940ed6ec59ea564"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.758072 14874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:05:47.758867 14896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:47.759104 14874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:47.759197 14874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1022331640fa4a76b6d0e8bcee763466"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "6748f9a4cbb65cae6c497f6f9e3e0769"
server_key_iv: "8ea49be1130760b2f940ed6ec59ea564"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.759253 14874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:47.787874 14874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:47.790088 14874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:47.790211 14874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:47.794423 14874 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:34941
I20260808 10:05:47.794462 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:34941 every 8 connection(s)
I20260808 10:05:47.794852 14874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:05:47.795866 14970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:47.798249 14970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap starting.
I20260808 10:05:47.798887 14970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:47.799189 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 14874
I20260808 10:05:47.799306 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:05:47.799324 14970 log.cc:824] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:47.799535 25226 external_mini_cluster.cc:1428] Setting key 4d62d38ee19c768446635545b4142d43
I20260808 10:05:47.800182 14970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: No bootstrap required, opened a new log
I20260808 10:05:47.802256 14970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:05:47.802433 14970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:47.802470 14970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Initialized, Role: FOLLOWER
I20260808 10:05:47.802616 14970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:05:47.802718 14970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:05:47.802754 14970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:05:47.802799 14970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:47.803503 14970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:05:47.803613 14970 leader_election.cc:302] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1022331640fa4a76b6d0e8bcee763466; no voters:
I20260808 10:05:47.803772 14970 leader_election.cc:288] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:05:47.803846 14975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:47.803997 14970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:05:47.804003 14975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 LEADER]: Becoming Leader. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Running, Role: LEADER
I20260808 10:05:47.804206 14975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:05:47.804613 14976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1022331640fa4a76b6d0e8bcee763466. Latest consensus state: current_term: 1 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:05:47.804840 14976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:47.804770 14977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:05:47.805298 14994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:05:47.805305 14977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:05:47.805927 14994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:05:47.808236 14994 catalog_manager.cc:1347] Generated new cluster ID: 6343652fa1c64d98b9ae47f44a0c2b43
I20260808 10:05:47.808286 14994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:05:47 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:47.812289 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:47.800118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.24.162.190:34941)
0808 10:05:47.800361 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:47.800374 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:47.800406 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:05:47.801439 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:47.801448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:47.801533 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:47.801690 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:47.802285 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:47.804038 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:47.805331 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:47.805886 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:47.806011 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:47.808915 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:47.808919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:47.808920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:47.808933 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:47.810701 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:47.811156 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:47.811159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:47.811160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:47.811265 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:47.811491 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:47.811491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:47.811492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:47.811729 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:47.811897 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:47.811994 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:05:47.812137 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":120,"threads_started":1}
I20260808 10:05:47.819443 14994 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:05:47.820143 14994 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:05:47.825747 14994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Generated new TSK 0
I20260808 10:05:47.825939 14994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:05:47.861618 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:47.961787 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:47.962080 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:47.962107 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:47.963405 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:47.963451 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:47.963478 15004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:05:47.964982 15004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:47.965214 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:47.965472 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:47.970520 15011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:47.970562 15010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:47.970783 15004 server_base.cc:1034] running on GCE node
W20260808 10:05:47.970526 15013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:47.971015 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:47.971266 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:47.972433 15004 hybrid_clock.cc:648] HybridClock initialized: now 1786183547972431 us; error 38 us; skew 500 ppm
Aug 08 10:05:47 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183547, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:47.974972 15004 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:05:47.975348 15004 webserver.cc:460] Webserver started at http://127.24.162.129:36183/ using document root <none> and password file <none>
I20260808 10:05:47.975543 15004 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:47.975582 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:47.975667 15004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:47.976635 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "faf82053fb4848dd9ce81f3687c6dc79"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "de97950d1026662fb725ad54de7fafd0"
server_key_iv: "4be8751ef6aecf2c111b723333fb1a2c"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.976944 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "faf82053fb4848dd9ce81f3687c6dc79"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "de97950d1026662fb725ad54de7fafd0"
server_key_iv: "4be8751ef6aecf2c111b723333fb1a2c"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.978258 15004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:05:47.978958 15024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:47.979168 15004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:47.979296 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "faf82053fb4848dd9ce81f3687c6dc79"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "de97950d1026662fb725ad54de7fafd0"
server_key_iv: "4be8751ef6aecf2c111b723333fb1a2c"
server_key_version: "encryptionkey@0"
I20260808 10:05:47.979382 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:47.996776 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:47.999056 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:47.999228 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:47.999485 15004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:47.999815 15004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:47.999872 15004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:47.999917 15004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:47.999943 15004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.006721 15004 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:33989
I20260808 10:05:48.006767 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:33989 every 8 connection(s)
I20260808 10:05:48.007171 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:05:48.008013 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15004
I20260808 10:05:48.008121 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:05:48.008364 25226 external_mini_cluster.cc:1428] Setting key f4bdbf273a0c4c059d0f877ef45585fa
Aug 08 10:05:48 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183547, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
I20260808 10:05:48.021818 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.007922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59307 (local address 127.24.162.190:34941)
0808 10:05:48.007971 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.007973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.008235 (+ 262us) server_negotiation.cc:407] Connection header received
0808 10:05:48.008792 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.008793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.008804 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.008867 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:48.009798 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.010558 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.012114 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.012419 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.012499 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.015561 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:48.015563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:48.015563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:48.015572 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:48.019731 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.020254 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.020255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.020255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.020334 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.020746 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.020747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.020748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.020942 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:48.021001 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.021510 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.021638 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:05:48.022554 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.007933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:59307)
0808 10:05:48.008116 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.008153 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.008694 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.008919 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.008922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.009508 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:48.009695 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.009700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.010686 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.010687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.012026 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.012030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.012576 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.012577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.012731 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.013331 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:48.013342 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:48.015425 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:48.019835 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.019845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.019848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.020150 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.020421 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.020422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.020422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.020616 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.021059 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:48.021068 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:48.021305 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.022073 (+ 768us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.022319 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260808 10:05:48.023320 15184 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:05:48.023442 15184 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:48.023664 15184 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
I20260808 10:05:48.024366 14923 ts_manager.cc:194] Registered new tserver with Master: faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989)
I20260808 10:05:48.025038 14923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:59307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:05:48.041419 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:48.123772 15189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:48.124015 15189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:48.124055 15189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:48.125217 15189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:48.125291 15189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:48.125339 15189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:05:48.126904 15189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:48.127197 15189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:48.127439 15189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:48.132571 15198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:48.132632 15195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:48.132723 15189 server_base.cc:1034] running on GCE node
W20260808 10:05:48.132572 15194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:48.132989 15189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:48.133198 15189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:48.134372 15189 hybrid_clock.cc:648] HybridClock initialized: now 1786183548134351 us; error 39 us; skew 500 ppm
Aug 08 10:05:48 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183548, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:48.137033 15189 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:05:48.137507 15189 webserver.cc:460] Webserver started at http://127.24.162.130:42079/ using document root <none> and password file <none>
I20260808 10:05:48.137755 15189 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:48.137800 15189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:48.137964 15189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:48.139115 15189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "dc16de8f52e1493bb52690ae2f96840c"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "ff56ed8f01aeb0e5aa9c5331e31dc2bb"
server_key_iv: "8f1aec306cb59cee19dbdbf5ee211f87"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.139467 15189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "dc16de8f52e1493bb52690ae2f96840c"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "ff56ed8f01aeb0e5aa9c5331e31dc2bb"
server_key_iv: "8f1aec306cb59cee19dbdbf5ee211f87"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.140815 15189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:05:48.141526 15209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.141724 15189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:05:48.141816 15189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "dc16de8f52e1493bb52690ae2f96840c"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "ff56ed8f01aeb0e5aa9c5331e31dc2bb"
server_key_iv: "8f1aec306cb59cee19dbdbf5ee211f87"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.141871 15189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:48.163897 15189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:48.166221 15189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:48.166347 15189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:48.166617 15189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:48.166986 15189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:48.167037 15189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.167075 15189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:48.167092 15189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.172736 15189 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:46659
I20260808 10:05:48.172822 15374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:46659 every 8 connection(s)
I20260808 10:05:48.173146 15189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:05:48.176666 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15189
I20260808 10:05:48.176793 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:05:48.177004 25226 external_mini_cluster.cc:1428] Setting key d57cc7a52b849acf80b6791bc937e891
Aug 08 10:05:48 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183548, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:48.183451 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.173900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:49393 (local address 127.24.162.190:34941)
0808 10:05:48.173965 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.173967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.174202 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:05:48.174731 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.174732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.174745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.174794 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:48.175814 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.176336 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.177587 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.177802 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.177889 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.180256 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:48.180258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:48.180259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:48.180268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:48.181925 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.182342 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.182343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.182344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.182412 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.182673 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.182674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.182675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.182815 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:48.182899 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.183252 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.183362 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:05:48.183669 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.173864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:49393)
0808 10:05:48.174054 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.174096 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.174605 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.174862 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.174866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.175495 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:48.175692 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.175698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.176452 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.176453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.177502 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.177505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.177901 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.177902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.178031 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.178566 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:48.178574 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:48.180163 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:48.182003 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.182010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.182014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.182238 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.182485 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.182486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.182487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.182594 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.182906 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:48.182934 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:48.183114 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.183229 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.183387 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260808 10:05:48.184204 15375 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:05:48.184317 15375 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:48.184521 15375 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
I20260808 10:05:48.184917 14923 ts_manager.cc:194] Registered new tserver with Master: dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
I20260808 10:05:48.185289 14923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:49393
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:05:48.205878 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:05:48.287263 15383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:05:48.287457 15383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:05:48.287480 15383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:05:48.288542 15383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:05:48.288578 15383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:05:48.288602 15383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:05:48.289942 15383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:05:48.290122 15383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:05:48.290369 15383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:05:48.295281 15389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:48.295324 15393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:05:48.295328 15390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:05:48.295526 15383 server_base.cc:1034] running on GCE node
I20260808 10:05:48.295689 15383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:05:48.295871 15383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:05:48.297006 15383 hybrid_clock.cc:648] HybridClock initialized: now 1786183548296995 us; error 27 us; skew 500 ppm
Aug 08 10:05:48 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183548, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:05:48.299441 15383 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:05:48.299898 15383 webserver.cc:460] Webserver started at http://127.24.162.131:45305/ using document root <none> and password file <none>
I20260808 10:05:48.300153 15383 fs_manager.cc:359] Metadata directory not provided
I20260808 10:05:48.300228 15383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:05:48.300383 15383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:05:48.301399 15383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "6c713adb186542f5988ce1e9227fc6f2"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "340ad8ac600c60da8ad986dcdf9d5a2b"
server_key_iv: "640f6f49dbd90239492425f46929abed"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.301785 15383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "6c713adb186542f5988ce1e9227fc6f2"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "340ad8ac600c60da8ad986dcdf9d5a2b"
server_key_iv: "640f6f49dbd90239492425f46929abed"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.303146 15383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:05:48.303915 15405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.304086 15383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:05:48.304206 15383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "6c713adb186542f5988ce1e9227fc6f2"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "340ad8ac600c60da8ad986dcdf9d5a2b"
server_key_iv: "640f6f49dbd90239492425f46929abed"
server_key_version: "encryptionkey@0"
I20260808 10:05:48.304327 15383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:05:48.320130 15383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:05:48.321918 15383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:05:48.322031 15383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:05:48.322214 15383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:05:48.322451 15383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:05:48.322481 15383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.322502 15383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:05:48.322515 15383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:05:48.328673 15383 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33575
I20260808 10:05:48.328727 15562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33575 every 8 connection(s)
I20260808 10:05:48.329027 15383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:05:48.331034 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15383
I20260808 10:05:48.331138 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:05:48.331332 25226 external_mini_cluster.cc:1428] Setting key 1e20f2864a264af0a0f3acf6f5b77001
Aug 08 10:05:48 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183548, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:05:48.338948 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.329727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:47051 (local address 127.24.162.190:34941)
0808 10:05:48.329797 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.329799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.329992 (+ 193us) server_negotiation.cc:407] Connection header received
0808 10:05:48.330459 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.330460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.330469 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.330526 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:48.331764 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.332394 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.333387 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.333605 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.333685 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.335959 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:48.335961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:48.335961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:48.335970 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:48.337426 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.337879 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.337880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.337880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.337942 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.338254 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.338255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.338255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.338379 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:48.338459 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.338740 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.338832 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:05:48.339136 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.329716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:47051)
0808 10:05:48.329868 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.329908 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.330380 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.330574 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.330577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.331404 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:05:48.331675 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.331682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.332477 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.332478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.333270 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.333272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.333709 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.333710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.333852 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.334425 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:05:48.334434 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:05:48.335863 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:05:48.337532 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.337539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.337541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.337786 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.338013 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:05:48.338014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:05:48.338015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:05:48.338153 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:05:48.338450 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:05:48.338457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:05:48.338599 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.338707 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.338870 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
I20260808 10:05:48.339615 15563 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:05:48.339711 15563 heartbeater.cc:464] Registering TS with master...
I20260808 10:05:48.339852 15563 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
I20260808 10:05:48.340201 14923 ts_manager.cc:194] Registered new tserver with Master: 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575)
I20260808 10:05:48.340539 14923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:47051
I20260808 10:05:48.343170 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:05:48.350181 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.344571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.24.162.190:34941)
0808 10:05:48.344610 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.344611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.344723 (+ 112us) server_negotiation.cc:407] Connection header received
0808 10:05:48.344767 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.344768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.344776 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.344823 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:05:48.345341 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.345884 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.346586 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.346811 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.346867 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.348058 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:05:48.348059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:05:48.348059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:05:48.348065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:05:48.349209 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.349534 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.349535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.349535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.349592 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:05:48.349765 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:05:48.349766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:05:48.349766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:05:48.349898 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:05:48.349975 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.350059 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.350121 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:05:48.351832 14923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:05:48.353427 14923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:05:48.359798 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.355721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:33989 (local address 127.0.0.1:58620)
0808 10:05:48.355990 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.356032 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.356099 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.356592 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.356592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.356594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:48.356640 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.356641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357875 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.357876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.358769 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.358772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359500 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.359582 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.359595 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.359668 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.359723 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":99,"threads_started":1}
I20260808 10:05:48.359812 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.355808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.24.162.129:33989)
0808 10:05:48.356084 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.356089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.356102 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:05:48.356232 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.356234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.356251 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.356307 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:48.356719 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357763 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.358860 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359394 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359498 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.359558 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.359690 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.359731 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
I20260808 10:05:48.359902 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.355865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33575 (local address 127.0.0.1:38086)
0808 10:05:48.356154 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.356166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.356244 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.356427 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.356428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.356430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:48.356498 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.356500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357419 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.357420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.358633 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.358636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359638 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.359720 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.359725 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.359774 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.359816 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":72,"threads_started":1}
I20260808 10:05:48.360159 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.356036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.24.162.131:33575)
0808 10:05:48.356216 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.356220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.356228 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:05:48.356310 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.356313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.356327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.356403 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:48.356603 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357313 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.358737 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359476 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359551 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.359600 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.360028 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.360078 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260808 10:05:48.360896 15474 tablet_service.cc:1467] Processing CreateTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 (DEFAULT_TABLE table=test-table [id=8e334701e6fe4567b07b500765407daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:48.360898 15088 tablet_service.cc:1467] Processing CreateTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 (DEFAULT_TABLE table=test-table [id=8e334701e6fe4567b07b500765407daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:48.361254 15474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bde84777bdd4d5c85d2ff842e72b854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:48.361254 15088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bde84777bdd4d5c85d2ff842e72b854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:48.363220 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.355664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46659 (local address 127.0.0.1:52788)
0808 10:05:48.355990 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.356033 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.356099 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.356418 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.356422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.356424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:48.356491 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.356493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357799 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.357800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.358833 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.358835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359803 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.362991 (+ 3187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.362996 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.363111 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.363155 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":61,"threads_started":1}
I20260808 10:05:48.363281 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.355754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.24.162.130:46659)
0808 10:05:48.355952 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.355958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.356184 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:05:48.356231 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.356234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.356250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.356322 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:48.356573 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.357686 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.358941 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.359657 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.359722 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.359771 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.363149 (+ 3378us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.363194 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
I20260808 10:05:48.364203 15277 tablet_service.cc:1467] Processing CreateTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 (DEFAULT_TABLE table=test-table [id=8e334701e6fe4567b07b500765407daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:05:48.364567 15277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bde84777bdd4d5c85d2ff842e72b854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:05:48.364702 15587 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap starting.
I20260808 10:05:48.364933 15588 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Bootstrap starting.
I20260808 10:05:48.365446 15587 tablet_bootstrap.cc:654] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:48.365520 15588 tablet_bootstrap.cc:654] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:48.365979 15588 log.cc:824] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:48.366046 15587 log.cc:824] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:48.366858 15588 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: No bootstrap required, opened a new log
I20260808 10:05:48.366971 15588 ts_tablet_manager.cc:1397] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:48.368028 15592 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Bootstrap starting.
I20260808 10:05:48.368577 15592 tablet_bootstrap.cc:654] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Neither blocks nor log segments found. Creating new log.
I20260808 10:05:48.368844 15588 raft_consensus.cc:348] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.368988 15588 raft_consensus.cc:374] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:48.369011 15588 raft_consensus.cc:729] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c713adb186542f5988ce1e9227fc6f2, State: Initialized, Role: FOLLOWER
I20260808 10:05:48.369120 15592 log.cc:824] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:05:48.369149 15588 consensus_queue.cc:260] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.369453 15588 ts_tablet_manager.cc:1428] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:05:48.369494 15563 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
I20260808 10:05:48.369942 15587 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: No bootstrap required, opened a new log
I20260808 10:05:48.370042 15587 ts_tablet_manager.cc:1397] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:05:48.370067 15592 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: No bootstrap required, opened a new log
I20260808 10:05:48.370136 15592 ts_tablet_manager.cc:1397] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:05:48.371701 15587 raft_consensus.cc:348] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.371722 15592 raft_consensus.cc:348] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.371861 15592 raft_consensus.cc:374] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:48.371882 15592 raft_consensus.cc:729] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc16de8f52e1493bb52690ae2f96840c, State: Initialized, Role: FOLLOWER
I20260808 10:05:48.371865 15587 raft_consensus.cc:374] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:05:48.371914 15587 raft_consensus.cc:729] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf82053fb4848dd9ce81f3687c6dc79, State: Initialized, Role: FOLLOWER
I20260808 10:05:48.372004 15592 consensus_queue.cc:260] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.372028 15587 consensus_queue.cc:260] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.372319 15587 ts_tablet_manager.cc:1428] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:05:48.372334 15375 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
I20260808 10:05:48.372335 15184 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
I20260808 10:05:48.372558 15592 ts_tablet_manager.cc:1428] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:05:48.390607 15595 raft_consensus.cc:482] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:05:48.390723 15595 raft_consensus.cc:504] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.391113 15595 leader_election.cc:288] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659), 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575)
I20260808 10:05:48.395112 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.391488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:47007 (local address 127.24.162.131:33575)
0808 10:05:48.391528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.391530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.391675 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:05:48.391715 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.391716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.391741 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.391798 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:48.391931 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.392599 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394136 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.394704 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394785 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.394794 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.394991 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.395038 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:05:48.395205 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.391344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:50177 (local address 127.24.162.130:46659)
0808 10:05:48.391395 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.391397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.391410 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:05:48.391492 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.391494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.391506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.391566 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:05:48.391766 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.392527 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394136 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.394716 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394785 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.394794 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.395093 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.395138 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:05:48.395272 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.391427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33575 (local address 127.24.162.129:47007)
0808 10:05:48.391599 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.391608 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.391665 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.391820 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.391821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.391822 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:48.391873 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.391874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.392682 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.392683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.394038 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.394042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.394805 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.394893 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.394901 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.394986 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.395044 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":95,"thread_start_us":112,"threads_started":1}
I20260808 10:05:48.395535 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.391289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46659 (local address 127.24.162.129:50177)
0808 10:05:48.391355 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.391368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:05:48.391427 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:05:48.391580 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:05:48.391581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:05:48.391583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:05:48.391690 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.391692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.392614 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.392614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.394039 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:05:48.394042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.394804 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.394806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:05:48.394893 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.394900 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:05:48.394986 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:05:48.395044 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":73}
I20260808 10:05:48.395665 15499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bde84777bdd4d5c85d2ff842e72b854" candidate_uuid: "faf82053fb4848dd9ce81f3687c6dc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c713adb186542f5988ce1e9227fc6f2" is_pre_election: true
I20260808 10:05:48.395931 15499 raft_consensus.cc:2454] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faf82053fb4848dd9ce81f3687c6dc79 in term 0.
I20260808 10:05:48.396023 15300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bde84777bdd4d5c85d2ff842e72b854" candidate_uuid: "faf82053fb4848dd9ce81f3687c6dc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc16de8f52e1493bb52690ae2f96840c" is_pre_election: true
I20260808 10:05:48.396190 15300 raft_consensus.cc:2454] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faf82053fb4848dd9ce81f3687c6dc79 in term 0.
I20260808 10:05:48.396204 15027 leader_election.cc:302] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c713adb186542f5988ce1e9227fc6f2, faf82053fb4848dd9ce81f3687c6dc79; no voters:
I20260808 10:05:48.396332 15595 raft_consensus.cc:2781] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:05:48.396412 15595 raft_consensus.cc:482] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:05:48.396437 15595 raft_consensus.cc:3037] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:48.397164 15595 raft_consensus.cc:504] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.397267 15595 leader_election.cc:288] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 1 election: Requested vote from peers dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659), 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575)
I20260808 10:05:48.397485 15300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bde84777bdd4d5c85d2ff842e72b854" candidate_uuid: "faf82053fb4848dd9ce81f3687c6dc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc16de8f52e1493bb52690ae2f96840c"
I20260808 10:05:48.397485 15499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bde84777bdd4d5c85d2ff842e72b854" candidate_uuid: "faf82053fb4848dd9ce81f3687c6dc79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c713adb186542f5988ce1e9227fc6f2"
I20260808 10:05:48.397567 15499 raft_consensus.cc:3037] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:48.397567 15300 raft_consensus.cc:3037] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:05:48.398463 15499 raft_consensus.cc:2454] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faf82053fb4848dd9ce81f3687c6dc79 in term 1.
I20260808 10:05:48.398463 15300 raft_consensus.cc:2454] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faf82053fb4848dd9ce81f3687c6dc79 in term 1.
I20260808 10:05:48.398660 15027 leader_election.cc:302] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c713adb186542f5988ce1e9227fc6f2, faf82053fb4848dd9ce81f3687c6dc79; no voters:
I20260808 10:05:48.398741 15595 raft_consensus.cc:2781] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:05:48.398933 15595 raft_consensus.cc:686] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 LEADER]: Becoming Leader. State: Replica: faf82053fb4848dd9ce81f3687c6dc79, State: Running, Role: LEADER
I20260808 10:05:48.399053 15595 consensus_queue.cc:237] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:05:48.399859 14923 catalog_manager.cc:5495] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 reported cstate change: term changed from 0 to 1, leader changed from <none> to faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "faf82053fb4848dd9ce81f3687c6dc79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } health_report { overall_health: UNKNOWN } } }
I20260808 10:05:48.407281 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.404805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58628 (local address 127.24.162.129:33989)
0808 10:05:48.404853 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:05:48.404856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:05:48.404880 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:05:48.404910 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:05:48.404911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:05:48.404928 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:05:48.405013 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:05:48.405147 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.405655 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.406411 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:05:48.406621 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:05:48.406702 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:05:48.406907 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:05:48.407026 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:05:48.407116 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:05:48.407164 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:05:48.407184 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:05:48.408995 15499 raft_consensus.cc:1261] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Refusing update from remote peer faf82053fb4848dd9ce81f3687c6dc79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:48.408995 15300 raft_consensus.cc:1261] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Refusing update from remote peer faf82053fb4848dd9ce81f3687c6dc79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:05:48.409310 15595 consensus_queue.cc:1035] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:48.409449 15600 consensus_queue.cc:1035] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:05:48.411969 15611 mvcc.cc:205] Tried to move back new op lower bound from 7316207814280437760 to 7316207814243098624. Current Snapshot: MvccSnapshot[applied={T|T < 7316207814280437760}]
I20260808 10:05:48.412647 15610 mvcc.cc:205] Tried to move back new op lower bound from 7316207814280437760 to 7316207814243098624. Current Snapshot: MvccSnapshot[applied={T|T < 7316207814280437760}]
I20260808 10:05:48.414975 14923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:48.415082 14923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:05:48.415850 14923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Sending DeleteTablet for 3 replicas of tablet 3bde84777bdd4d5c85d2ff842e72b854
I20260808 10:05:48.416208 15088 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:48 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260808 10:05:48.416213 15277 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:48 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:52788
I20260808 10:05:48.416239 15474 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:48 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:38086
I20260808 10:05:48.416505 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15004
I20260808 10:05:48.425448 15621 tablet_replica.cc:330] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: stopping tablet replica
I20260808 10:05:48.425578 15621 raft_consensus.cc:2229] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:48.425681 15621 raft_consensus.cc:2258] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:48.426990 15622 tablet_replica.cc:330] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: stopping tablet replica
I20260808 10:05:48.426994 15621 ts_tablet_manager.cc:1905] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:48.427119 15622 raft_consensus.cc:2229] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:05:48.427228 15622 raft_consensus.cc:2258] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:05:48.427702 15622 ts_tablet_manager.cc:1905] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:05:48.428714 15621 ts_tablet_manager.cc:1918] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:48.428803 15621 log.cc:1196] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/3bde84777bdd4d5c85d2ff842e72b854
I20260808 10:05:48.429068 15621 ts_tablet_manager.cc:1939] T 3bde84777bdd4d5c85d2ff842e72b854 P dc16de8f52e1493bb52690ae2f96840c: Deleting consensus metadata
I20260808 10:05:48.429534 14900 catalog_manager.cc:4841] TS dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): tablet 3bde84777bdd4d5c85d2ff842e72b854 (table test-table [id=8e334701e6fe4567b07b500765407daf]) successfully deleted
I20260808 10:05:48.429637 15622 ts_tablet_manager.cc:1918] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:05:48.429716 15622 log.cc:1196] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/3bde84777bdd4d5c85d2ff842e72b854
I20260808 10:05:48.429998 15622 ts_tablet_manager.cc:1939] T 3bde84777bdd4d5c85d2ff842e72b854 P 6c713adb186542f5988ce1e9227fc6f2: Deleting consensus metadata
I20260808 10:05:48.430434 14900 catalog_manager.cc:4841] TS 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): tablet 3bde84777bdd4d5c85d2ff842e72b854 (table test-table [id=8e334701e6fe4567b07b500765407daf]) successfully deleted
W20260808 10:05:48.431969 14899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:33989 (error 108)
I20260808 10:05:48.432237 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15189
I20260808 10:05:48.432541 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:05:48.432415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:33989 (local address 127.0.0.1:58636)
0808 10:05:48.432466 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:05:48.432502 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:05:48.433811 14899 catalog_manager.cc:4584] TS faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bde84777bdd4d5c85d2ff842e72b854: Network error: Client connection negotiation failed: client connection to 127.24.162.129:33989: connect: Connection refused (error 111)
I20260808 10:05:48.439798 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15383
I20260808 10:05:48.449240 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 14874
I20260808 10:06:08.457261 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--webserver_interface=127.24.162.190
--webserver_port=35281
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:34941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:08.540238 15624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:08.540454 15624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:08.540478 15624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:08.541693 15624 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:06:08.541739 15624 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:06:08.541754 15624 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:06:08.541766 15624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:08.541780 15624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:08.541793 15624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:08.541847 15624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:08.541877 15624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:08.543438 15624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:34941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:08.543735 15624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:08.544011 15624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:08.553848 15638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.553848 15634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.553890 15633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.554222 15624 server_base.cc:1034] running on GCE node
I20260808 10:06:08.554502 15624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:08.554826 15624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:08.556007 15624 hybrid_clock.cc:648] HybridClock initialized: now 1786183568555983 us; error 42 us; skew 500 ppm
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:08.558615 15624 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:06:08.559139 15624 webserver.cc:460] Webserver started at http://127.24.162.190:35281/ using document root <none> and password file <none>
I20260808 10:06:08.559361 15624 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:08.559418 15624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:08.560778 15624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:08.561457 15649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.561638 15624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:08.561766 15624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1022331640fa4a76b6d0e8bcee763466"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "6748f9a4cbb65cae6c497f6f9e3e0769"
server_key_iv: "8ea49be1130760b2f940ed6ec59ea564"
server_key_version: "encryptionkey@0"
I20260808 10:06:08.562150 15624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:08.571327 15624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:08.573403 15624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:08.573518 15624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:08.577649 15624 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:34941
I20260808 10:06:08.577740 15720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:34941 every 8 connection(s)
I20260808 10:06:08.578116 15624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:08.579743 15721 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:06:08.580353 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap starting.
I20260808 10:06:08.583412 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15624
I20260808 10:06:08.583760 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:33989
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=36183
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:06:08.589810 15721 log.cc:824] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:08.591867 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:06:08.592182 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap complete.
I20260808 10:06:08.594456 15721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:08.594719 15721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Initialized, Role: FOLLOWER
I20260808 10:06:08.594945 15721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:08.595044 15721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:08.595106 15721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:08.595155 15721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:06:08.596145 15721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:08.596271 15721 leader_election.cc:302] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1022331640fa4a76b6d0e8bcee763466; no voters:
I20260808 10:06:08.596473 15721 leader_election.cc:288] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:06:08.596511 15728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:06:08.596637 15728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 LEADER]: Becoming Leader. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Running, Role: LEADER
I20260808 10:06:08.596757 15721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:08.596791 15728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:08.597172 15729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:06:08.597275 15729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:08.597666 15732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1022331640fa4a76b6d0e8bcee763466. Latest consensus state: current_term: 2 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:06:08.597738 15732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:08.597903 15738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:08.598747 15738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:08.598856 15738 catalog_manager.cc:1259] Loaded cluster ID: 6343652fa1c64d98b9ae47f44a0c2b43
I20260808 10:06:08.598973 15738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:08.599939 15738 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:08.600154 15738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Loaded TSK: 0
I20260808 10:06:08.612727 15738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Generated new TSK 1
I20260808 10:06:08.612859 15738 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:06:08.691423 15724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:08.691623 15724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:08.691648 15724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:08.692811 15724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:08.692860 15724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:08.692888 15724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:08.694348 15724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:33989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=36183
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:08.694600 15724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:08.694869 15724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:08.700999 15768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.701042 15766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.701244 15724 server_base.cc:1034] running on GCE node
W20260808 10:06:08.701300 15764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.701541 15724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:08.701786 15724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:08.702992 15724 hybrid_clock.cc:648] HybridClock initialized: now 1786183568702977 us; error 33 us; skew 500 ppm
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:08.705487 15724 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:06:08.705932 15724 webserver.cc:460] Webserver started at http://127.24.162.129:36183/ using document root <none> and password file <none>
I20260808 10:06:08.706188 15724 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:08.706282 15724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:08.707808 15724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:08.708561 15778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.708760 15724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:08.708885 15724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "faf82053fb4848dd9ce81f3687c6dc79"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "de97950d1026662fb725ad54de7fafd0"
server_key_iv: "4be8751ef6aecf2c111b723333fb1a2c"
server_key_version: "encryptionkey@0"
I20260808 10:06:08.709233 15724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:08.716728 15724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:08.719245 15724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:08.719417 15724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:08.719657 15724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:08.720125 15787 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:06:08.722177 15724 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:06:08.722242 15724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:06:08.722296 15724 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:06:08.723048 15724 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:06:08.723105 15724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:08.723165 15787 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap starting.
I20260808 10:06:08.730679 15724 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:33989
I20260808 10:06:08.731161 15724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:08.734390 15787 log.cc:824] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:08.735596 15930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:33989 every 8 connection(s)
I20260808 10:06:08.737102 15787 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:06:08.737432 15787 tablet_bootstrap.cc:492] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap complete.
I20260808 10:06:08.738736 15787 ts_tablet_manager.cc:1397] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.012s
I20260808 10:06:08.740105 15787 raft_consensus.cc:348] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:06:08.740461 15787 raft_consensus.cc:729] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf82053fb4848dd9ce81f3687c6dc79, State: Initialized, Role: FOLLOWER
I20260808 10:06:08.740600 15787 consensus_queue.cc:260] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } } peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } }
I20260808 10:06:08.740754 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15724
I20260808 10:06:08.740921 15787 ts_tablet_manager.cc:1428] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260808 10:06:08.741094 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:46659
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=42079
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:08.746503 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.731988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:47245 (local address 127.24.162.190:34941)
0808 10:06:08.734848 (+ 2860us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:08.734858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:08.734906 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:06:08.735567 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:08.735574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:08.735637 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:08.735906 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:08.736565 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.737613 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.738662 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.739197 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.739334 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.742541 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:08.742546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:08.742546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:08.742561 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:08.744674 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.745189 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.745192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.745192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.745283 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.745572 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.745573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.745573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.745799 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:08.745867 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:08.746219 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 10:06:08.746337 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2794,"thread_start_us":79,"threads_started":1}
I20260808 10:06:08.746531 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.731890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:47245)
0808 10:06:08.734717 (+ 2827us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:08.734767 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:08.735323 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:08.735939 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:08.735943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:08.736241 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:08.736445 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.736450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.737719 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.737721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.738552 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.738554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.739317 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.739319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.739483 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.740082 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:08.740091 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:08.742419 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:08.744761 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.744769 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.744771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.745091 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.745354 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.745355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.745355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.745498 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.745877 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:08.745885 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:08.746039 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 10:06:08.746162 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:06:08.746335 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2774,"thread_start_us":101,"threads_started":1}
I20260808 10:06:08.747403 15936 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:06:08.747504 15936 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:08.747674 15936 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
I20260808 10:06:08.748396 15671 ts_manager.cc:194] Registered new tserver with Master: faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989)
I20260808 10:06:08.749536 15671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:47245
I20260808 10:06:08.750586 15936 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.129@KRBTEST.COM
I20260808 10:06:08.758476 15949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.749814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.24.162.129:33989)
0808 10:06:08.749960 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:08.749964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:08.749974 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:08.750007 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:08.750009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:08.750021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:08.750699 (+ 678us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:08.750752 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.751574 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.752943 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.753218 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.753267 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.755595 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:08.755604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:08.755604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:08.755611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:08.757180 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.757597 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.757600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.757600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.757662 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.757875 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.757876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.757876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.758010 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:08.758098 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:08.758289 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 10:06:08.758374 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260808 10:06:08.758486 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.749058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:33989 (local address 127.0.0.1:36372)
0808 10:06:08.749271 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:08.749308 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:08.749371 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:08.750128 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:08.750135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:08.750474 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:08.750565 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.750568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.751654 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.751655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.752848 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.752852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.753292 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.753293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.753405 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.753880 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:08.753887 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:08.755502 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:08.757290 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.757298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.757300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.757508 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.757735 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.757736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.757736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.757802 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.758091 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:08.758098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:08.758151 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 10:06:08.758249 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:06:08.758374 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":136,"threads_started":1}
I20260808 10:06:08.759361 15825 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bde84777bdd4d5c85d2ff842e72b854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:05:48 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:36372
I20260808 10:06:08.759920 15950 tablet_replica.cc:330] stopping tablet replica
I20260808 10:06:08.760052 15950 raft_consensus.cc:2229] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:08.760126 15950 raft_consensus.cc:2258] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:08.760452 15950 ts_tablet_manager.cc:1905] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:08.762393 15950 ts_tablet_manager.cc:1918] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:08.762486 15950 log.cc:1196] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal/wals/3bde84777bdd4d5c85d2ff842e72b854
I20260808 10:06:08.762723 15950 ts_tablet_manager.cc:1939] T 3bde84777bdd4d5c85d2ff842e72b854 P faf82053fb4848dd9ce81f3687c6dc79: Deleting consensus metadata
I20260808 10:06:08.763312 15651 catalog_manager.cc:4841] TS faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989): tablet 3bde84777bdd4d5c85d2ff842e72b854 (table test-table [id=8e334701e6fe4567b07b500765407daf]) successfully deleted
W20260808 10:06:08.839895 15946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:08.840094 15946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:08.840116 15946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:08.841313 15946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:08.841359 15946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:08.841419 15946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:08.842888 15946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:46659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=42079
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:08.843252 15946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:08.843524 15946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:08.848668 15956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.848731 15946 server_base.cc:1034] running on GCE node
W20260808 10:06:08.848649 15957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.848649 15960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.849010 15946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:08.849278 15946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:08.850441 15946 hybrid_clock.cc:648] HybridClock initialized: now 1786183568850442 us; error 34 us; skew 500 ppm
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:08.852880 15946 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:06:08.853367 15946 webserver.cc:460] Webserver started at http://127.24.162.130:42079/ using document root <none> and password file <none>
I20260808 10:06:08.853621 15946 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:08.853708 15946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:08.855110 15946 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:08.855717 15968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.855885 15946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.855990 15946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "dc16de8f52e1493bb52690ae2f96840c"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "ff56ed8f01aeb0e5aa9c5331e31dc2bb"
server_key_iv: "8f1aec306cb59cee19dbdbf5ee211f87"
server_key_version: "encryptionkey@0"
I20260808 10:06:08.856353 15946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:08.871726 15946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:08.873505 15946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:08.873673 15946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:08.873906 15946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:08.874219 15946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:08.874275 15946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.874316 15946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:08.874341 15946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.880272 15946 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:46659
I20260808 10:06:08.880378 16125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:46659 every 8 connection(s)
I20260808 10:06:08.880728 15946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:08.889725 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.881504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:48589 (local address 127.24.162.190:34941)
0808 10:06:08.881573 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:08.881575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:08.881746 (+ 171us) server_negotiation.cc:407] Connection header received
0808 10:06:08.882203 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:08.882204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:08.882213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:08.882267 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:08.883265 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.883767 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.884741 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.884949 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.885007 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.887201 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:08.887202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:08.887202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:08.887209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:08.888598 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.888970 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.888971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.888971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.889020 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:08.889226 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:08.889226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:08.889227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:08.889324 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:08.889376 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:08.889594 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 10:06:08.889658 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:06:08.890000 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:08.881433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:48589)
0808 10:06:08.881598 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:08.881640 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:08.882119 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:08.882324 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:08.882327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:08.882887 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:08.883145 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.883149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.883859 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.883860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.884657 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:08.884659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:08.885028 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:08.885029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:08.885125 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:08.885639 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:08.885647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:08.887110 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:08.888667 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.888672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.888674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.888905 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.889079 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:08.889080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:08.889080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:08.889172 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:08.889380 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:08.889384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:08.889490 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 10:06:08.889571 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:06:08.889718 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260808 10:06:08.890574 16127 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:06:08.890657 16127 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:08.890856 16127 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
I20260808 10:06:08.890964 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 15946
I20260808 10:06:08.891286 15671 ts_manager.cc:194] Registered new tserver with Master: dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
I20260808 10:06:08.891304 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:33575
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=45305
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:06:08.891700 15671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:48589
W20260808 10:06:08.973809 16130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:08.974038 16130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:08.974076 16130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:08.975176 16130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:08.975240 16130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:08.975281 16130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:08.976716 16130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:33575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=45305
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:08.976984 16130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:08.977229 16130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:08.982293 16136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.982302 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:08.982303 16135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:08.982463 16130 server_base.cc:1034] running on GCE node
I20260808 10:06:08.982728 16130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:08.982981 16130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:08.984133 16130 hybrid_clock.cc:648] HybridClock initialized: now 1786183568984133 us; error 38 us; skew 500 ppm
Aug 08 10:06:08 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:08.986541 16130 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:06:08.987028 16130 webserver.cc:460] Webserver started at http://127.24.162.131:45305/ using document root <none> and password file <none>
I20260808 10:06:08.987262 16130 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:08.987342 16130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:08.988653 16130 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:08.989239 16147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.989423 16130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:08.989523 16130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "6c713adb186542f5988ce1e9227fc6f2"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "340ad8ac600c60da8ad986dcdf9d5a2b"
server_key_iv: "640f6f49dbd90239492425f46929abed"
server_key_version: "encryptionkey@0"
I20260808 10:06:08.989856 16130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:09.021402 16130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:09.023963 16130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:09.024138 16130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:09.024400 16130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:09.024724 16130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:09.024781 16130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:09.024818 16130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:09.024868 16130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:09.030616 16130 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33575
I20260808 10:06:09.030699 16301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33575 every 8 connection(s)
I20260808 10:06:09.031101 16130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:06:09 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183568, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:09.037129 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 16130
I20260808 10:06:09.041179 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.031899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:40679 (local address 127.24.162.190:34941)
0808 10:06:09.031959 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.031961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.032171 (+ 210us) server_negotiation.cc:407] Connection header received
0808 10:06:09.032622 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.032623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.032633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.032688 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:09.033597 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.034082 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.035156 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.035346 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.035402 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.037795 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:09.037796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:09.037797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:09.037804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:09.039602 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.040063 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.040063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.040064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.040116 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.040413 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.040414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.040414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.040540 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:09.040619 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.040974 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.041065 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:06:09.041369 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.031878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:40679)
0808 10:06:09.032042 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.032079 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.032538 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.032746 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:09.032749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:09.033307 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:09.033494 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.033498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.034162 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.034162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.035077 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.035080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.035418 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.035419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.035516 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.036045 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:09.036054 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:09.037679 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:09.039681 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:09.039688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:09.039690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:09.039985 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:09.040182 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:09.040183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:09.040183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:09.040344 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:09.040647 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:09.040654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:09.040815 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 10:06:09.040941 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 10:06:09.041099 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260808 10:06:09.041927 16302 heartbeater.cc:347] Connected to a master server at 127.24.162.190:34941
I20260808 10:06:09.042037 16302 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:09.042217 16302 heartbeater.cc:511] Master 127.24.162.190:34941 requested a full tablet report, sending...
I20260808 10:06:09.042634 15671 ts_manager.cc:194] Registered new tserver with Master: 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575)
I20260808 10:06:09.043092 15671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:40679
I20260808 10:06:09.044831 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.037563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.24.162.190:34941)
0808 10:06:09.037689 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.037691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.037784 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:06:09.037848 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.037849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.037861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.037937 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:09.038490 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.039300 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.040300 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.040566 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.040628 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.042086 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:09.042088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:09.042088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:09.042095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:09.043643 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.044026 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.044027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.044027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.044084 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.044303 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.044304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.044304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.044424 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:09.044615 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.044653 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.044732 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260808 10:06:09.045594 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:09.047955 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.045927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.24.162.190:34941)
0808 10:06:09.045960 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.045961 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.046062 (+ 101us) server_negotiation.cc:407] Connection header received
0808 10:06:09.046108 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.046109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.046134 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.046196 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:09.046264 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.046684 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.047410 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.047588 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.047641 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.047759 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:09.047795 (+ 36us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 10:06:09.047916 (+ 121us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35696: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 10:06:09.048023 16309 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35696: token expired
W20260808 10:06:09.048336 15569 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.162.190:34941)
I20260808 10:06:09.054033 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.048675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.24.162.190:34941)
0808 10:06:09.048750 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.048751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.048758 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:06:09.048782 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.048783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.048790 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.048856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:09.049313 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.049849 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.050626 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.050812 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.050867 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.051999 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:09.052000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:09.052000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:09.052005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:09.053148 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.053457 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.053458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.053458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.053504 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:09.053678 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:09.053678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:09.053678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:09.053764 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:09.053828 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.053893 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.053947 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:06:09.055178 15671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:09.056758 15671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:09.059078 15825 tablet_service.cc:1467] Processing CreateTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 (DEFAULT_TABLE table=test-table [id=c441122b9a344c9dacc008a236c059b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:09.059290 15825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f47b96221e4d96aa6f623d90f93d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:09.060716 15787 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap starting.
I20260808 10:06:09.061054 15787 tablet_bootstrap.cc:654] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:09.061932 15787 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: No bootstrap required, opened a new log
I20260808 10:06:09.062000 15787 ts_tablet_manager.cc:1397] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:09.062151 15787 raft_consensus.cc:348] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.062245 15787 raft_consensus.cc:374] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:09.062294 15787 raft_consensus.cc:729] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf82053fb4848dd9ce81f3687c6dc79, State: Initialized, Role: FOLLOWER
I20260808 10:06:09.062345 15787 consensus_queue.cc:260] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.062433 15787 ts_tablet_manager.cc:1428] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:09.062860 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.058763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46659 (local address 127.0.0.1:42002)
0808 10:06:09.058974 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.058985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.059034 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.059350 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:09.059351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:09.059354 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:09.059397 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.059398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.060441 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.060442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.061703 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.061705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.062599 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.062601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.062663 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.062685 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:06:09.062750 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:06:09.062789 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260808 10:06:09.062876 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.058888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.24.162.130:46659)
0808 10:06:09.059139 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.059146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.059178 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:06:09.059219 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.059221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.059235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.059332 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:09.059456 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.060361 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.061787 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.062518 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.062578 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.062614 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.062769 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.062800 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":119,"threads_started":1}
I20260808 10:06:09.063584 16029 tablet_service.cc:1467] Processing CreateTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 (DEFAULT_TABLE table=test-table [id=c441122b9a344c9dacc008a236c059b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:09.063920 16029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f47b96221e4d96aa6f623d90f93d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:09.064345 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.058659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33575 (local address 127.0.0.1:48804)
0808 10:06:09.058692 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.058702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.058736 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.060961 (+ 2225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:09.060962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:09.060964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:09.060998 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.060999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.062216 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.062217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.063309 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.063312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.064152 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.064152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.064223 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.064226 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:09.064270 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 10:06:09.064303 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 10:06:09.064409 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.058830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.24.162.131:33575)
0808 10:06:09.060809 (+ 1979us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.060813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.060823 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:09.060863 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.060865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.060880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.060965 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:09.061092 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.062131 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.063390 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.064079 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.064156 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.064204 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.064283 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.064323 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1945,"thread_start_us":136,"threads_started":1}
I20260808 10:06:09.064986 16212 tablet_service.cc:1467] Processing CreateTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 (DEFAULT_TABLE table=test-table [id=c441122b9a344c9dacc008a236c059b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:09.065310 16212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f47b96221e4d96aa6f623d90f93d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:09.067607 16320 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Bootstrap starting.
I20260808 10:06:09.068187 16320 tablet_bootstrap.cc:654] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:09.068579 16322 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Bootstrap starting.
I20260808 10:06:09.068769 16320 log.cc:824] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:09.069157 16322 tablet_bootstrap.cc:654] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:09.069617 16320 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: No bootstrap required, opened a new log
I20260808 10:06:09.069697 16320 ts_tablet_manager.cc:1397] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:09.069746 16322 log.cc:824] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:09.070608 16322 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: No bootstrap required, opened a new log
I20260808 10:06:09.070683 16322 ts_tablet_manager.cc:1397] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:09.071704 16320 raft_consensus.cc:348] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.071907 16320 raft_consensus.cc:374] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:09.071949 16320 raft_consensus.cc:729] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc16de8f52e1493bb52690ae2f96840c, State: Initialized, Role: FOLLOWER
I20260808 10:06:09.072072 16320 consensus_queue.cc:260] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.072360 16320 ts_tablet_manager.cc:1428] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:09.072376 16127 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
I20260808 10:06:09.072649 16322 raft_consensus.cc:348] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.072860 16322 raft_consensus.cc:374] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:09.072916 16322 raft_consensus.cc:729] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c713adb186542f5988ce1e9227fc6f2, State: Initialized, Role: FOLLOWER
I20260808 10:06:09.073024 16322 consensus_queue.cc:260] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.073324 16322 ts_tablet_manager.cc:1428] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:09.073334 16302 heartbeater.cc:503] Master 127.24.162.190:34941 was elected leader, sending a full tablet report...
W20260808 10:06:09.131371 16128 tablet.cc:2369] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:09.231637 16327 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:09.231771 16327 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.232214 16327 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989)
W20260808 10:06:09.235065 15940 tablet.cc:2369] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:09.236083 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.232497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:34861 (local address 127.24.162.131:33575)
0808 10:06:09.232628 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.232630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.232721 (+ 91us) server_negotiation.cc:407] Connection header received
0808 10:06:09.232749 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.232751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.232760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.232811 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:09.232971 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.233716 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235322 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.235693 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235759 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.235766 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.235956 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.236003 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260808 10:06:09.236111 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.232447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33575 (local address 127.24.162.130:34861)
0808 10:06:09.232619 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.232635 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.232681 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.232828 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:09.232828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:09.232830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:09.232892 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.232893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.233845 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.233846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.235167 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.235169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.235792 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.235871 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.235876 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:06:09.235933 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:06:09.235982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":36,"thread_start_us":113,"threads_started":1}
I20260808 10:06:09.236187 15949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.232512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:53439 (local address 127.24.162.129:33989)
0808 10:06:09.232583 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.232586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.232598 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:06:09.232637 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.232639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.232672 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.232755 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:09.232975 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.233719 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235278 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.235787 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235866 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.235948 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.236071 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.236116 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:06:09.236372 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.232424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:33989 (local address 127.24.162.130:53439)
0808 10:06:09.232484 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.232498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.232542 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.232779 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:09.232780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:09.232782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:09.232876 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.232879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.233844 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.233846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.235166 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:09.235169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.235870 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.235870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:09.235981 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.235985 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:09.236044 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:06:09.236081 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":69}
I20260808 10:06:09.236618 16238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f47b96221e4d96aa6f623d90f93d38" candidate_uuid: "dc16de8f52e1493bb52690ae2f96840c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c713adb186542f5988ce1e9227fc6f2" is_pre_election: true
I20260808 10:06:09.236829 16238 raft_consensus.cc:2454] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc16de8f52e1493bb52690ae2f96840c in term 0.
I20260808 10:06:09.236763 15870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f47b96221e4d96aa6f623d90f93d38" candidate_uuid: "dc16de8f52e1493bb52690ae2f96840c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faf82053fb4848dd9ce81f3687c6dc79" is_pre_election: true
I20260808 10:06:09.236951 15870 raft_consensus.cc:2454] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc16de8f52e1493bb52690ae2f96840c in term 0.
I20260808 10:06:09.237079 15972 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c; no voters:
I20260808 10:06:09.237267 16327 raft_consensus.cc:2781] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:09.237332 16327 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:09.237376 16327 raft_consensus.cc:3037] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:09.238106 16327 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.238260 16327 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [CANDIDATE]: Term 1 election: Requested vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989)
I20260808 10:06:09.238420 15870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f47b96221e4d96aa6f623d90f93d38" candidate_uuid: "dc16de8f52e1493bb52690ae2f96840c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faf82053fb4848dd9ce81f3687c6dc79"
I20260808 10:06:09.238432 16238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f47b96221e4d96aa6f623d90f93d38" candidate_uuid: "dc16de8f52e1493bb52690ae2f96840c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c713adb186542f5988ce1e9227fc6f2"
I20260808 10:06:09.238503 15870 raft_consensus.cc:3037] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:09.238503 16238 raft_consensus.cc:3037] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:09.239440 15870 raft_consensus.cc:2454] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc16de8f52e1493bb52690ae2f96840c in term 1.
I20260808 10:06:09.239440 16238 raft_consensus.cc:2454] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc16de8f52e1493bb52690ae2f96840c in term 1.
I20260808 10:06:09.239636 15970 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc16de8f52e1493bb52690ae2f96840c, faf82053fb4848dd9ce81f3687c6dc79; no voters:
I20260808 10:06:09.239778 16327 raft_consensus.cc:2781] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:09.239972 16327 raft_consensus.cc:686] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 1 LEADER]: Becoming Leader. State: Replica: dc16de8f52e1493bb52690ae2f96840c, State: Running, Role: LEADER
I20260808 10:06:09.240078 16327 consensus_queue.cc:237] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:09.240911 15671 catalog_manager.cc:5495] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c reported cstate change: term changed from 0 to 1, leader changed from <none> to dc16de8f52e1493bb52690ae2f96840c (127.24.162.130). New cstate: current_term: 1 leader_uuid: "dc16de8f52e1493bb52690ae2f96840c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } health_report { overall_health: UNKNOWN } } }
W20260808 10:06:09.281813 16303 tablet.cc:2369] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:09.288167 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:09.285719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.24.162.130:46659)
0808 10:06:09.285781 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:09.285783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:09.285794 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:09.285851 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:09.285852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:09.285872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:09.285954 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:09.286065 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.286539 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.287373 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:09.287558 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:09.287630 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:09.287804 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:09.287908 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:09.288008 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:09.288028 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 10:06:09.288049 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:06:09.289999 15870 raft_consensus.cc:1261] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Refusing update from remote peer dc16de8f52e1493bb52690ae2f96840c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:09.290021 16238 raft_consensus.cc:1261] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Refusing update from remote peer dc16de8f52e1493bb52690ae2f96840c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:09.290427 16332 consensus_queue.cc:1035] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:09.290571 16327 consensus_queue.cc:1035] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [LEADER]: Connected to new peer: Peer: permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:09.296010 15670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:09.296126 15670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:09.296967 15670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Sending DeleteTablet for 3 replicas of tablet 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:09.297258 15825 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:36372
I20260808 10:06:09.297363 16029 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:42002
I20260808 10:06:09.297443 16212 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f47b96221e4d96aa6f623d90f93d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:09 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:48804
I20260808 10:06:09.297510 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15724
I20260808 10:06:09.307760 16349 tablet_replica.cc:330] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: stopping tablet replica
I20260808 10:06:09.307911 16349 raft_consensus.cc:2229] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:09.308069 16349 raft_consensus.cc:2258] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:09.308279 16352 tablet_replica.cc:330] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: stopping tablet replica
I20260808 10:06:09.308395 16352 raft_consensus.cc:2229] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:09.308504 16352 raft_consensus.cc:2258] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:09.309268 16349 ts_tablet_manager.cc:1905] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:09.309577 16352 ts_tablet_manager.cc:1905] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:09.311671 16352 ts_tablet_manager.cc:1918] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:09.311825 16352 log.cc:1196] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:09.312203 16352 ts_tablet_manager.cc:1939] T 19f47b96221e4d96aa6f623d90f93d38 P 6c713adb186542f5988ce1e9227fc6f2: Deleting consensus metadata
I20260808 10:06:09.312775 15652 catalog_manager.cc:4841] TS 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): tablet 19f47b96221e4d96aa6f623d90f93d38 (table test-table [id=c441122b9a344c9dacc008a236c059b7]) successfully deleted
I20260808 10:06:09.312836 16349 ts_tablet_manager.cc:1918] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:09.312911 16349 log.cc:1196] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:09.313026 15651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:33989 (error 108)
I20260808 10:06:09.313212 16349 ts_tablet_manager.cc:1939] T 19f47b96221e4d96aa6f623d90f93d38 P dc16de8f52e1493bb52690ae2f96840c: Deleting consensus metadata
W20260808 10:06:09.313714 15948 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:06:09.313490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:33989 (local address 127.0.0.1:36376)
0808 10:06:09.313554 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:09.313567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:09.313618 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:09.313665 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:33989: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 10:06:09.313766 15652 catalog_manager.cc:4841] TS dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): tablet 19f47b96221e4d96aa6f623d90f93d38 (table test-table [id=c441122b9a344c9dacc008a236c059b7]) successfully deleted
W20260808 10:06:09.313835 15651 catalog_manager.cc:4584] TS faf82053fb4848dd9ce81f3687c6dc79 (127.24.162.129:33989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19f47b96221e4d96aa6f623d90f93d38: Network error: Client connection negotiation failed: client connection to 127.24.162.129:33989: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 10:06:09.314508 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15946
I20260808 10:06:09.320935 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 16130
I20260808 10:06:09.330606 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 15624
I20260808 10:06:29.338011 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--webserver_interface=127.24.162.190
--webserver_port=35281
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:34941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:29.422273 16355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:29.422495 16355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:29.422518 16355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:29.423695 16355 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:06:29.423736 16355 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:06:29.423749 16355 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:06:29.423761 16355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:29.423775 16355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:29.423833 16355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:29.423851 16355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:29.423874 16355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:29.425329 16355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:34941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:34941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:29.425578 16355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:29.425823 16355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:29.431051 16361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.431074 16360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.431058 16365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:29.431134 16355 server_base.cc:1034] running on GCE node
I20260808 10:06:29.431497 16355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:29.431742 16355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:29.432927 16355 hybrid_clock.cc:648] HybridClock initialized: now 1786183589432921 us; error 37 us; skew 500 ppm
Aug 08 10:06:29 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:29.435530 16355 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:06:29.435954 16355 webserver.cc:460] Webserver started at http://127.24.162.190:35281/ using document root <none> and password file <none>
I20260808 10:06:29.436182 16355 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:29.436254 16355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:29.437594 16355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:29.438249 16376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.438417 16355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:29.438524 16355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "1022331640fa4a76b6d0e8bcee763466"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "6748f9a4cbb65cae6c497f6f9e3e0769"
server_key_iv: "8ea49be1130760b2f940ed6ec59ea564"
server_key_version: "encryptionkey@0"
I20260808 10:06:29.438862 16355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:29.450042 16355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:29.452399 16355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:29.452571 16355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:29.457013 16355 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:34941
I20260808 10:06:29.457072 16447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:34941 every 8 connection(s)
I20260808 10:06:29.457471 16355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:29.458959 16448 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:06:29.459652 16448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap starting.
I20260808 10:06:29.463871 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 16355
I20260808 10:06:29.464154 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:33989
--local_ip_for_outbound_sockets=127.24.162.129
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=36183
--webserver_interface=127.24.162.129
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:06:29.469054 16448 log.cc:824] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:29.471388 16448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:06:29.471725 16448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Bootstrap complete.
I20260808 10:06:29.474042 16448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:29.474261 16448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Initialized, Role: FOLLOWER
I20260808 10:06:29.474442 16448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:29.474504 16448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:29.474565 16448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:29.474614 16448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:06:29.475697 16448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:29.475787 16448 leader_election.cc:302] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1022331640fa4a76b6d0e8bcee763466; no voters:
I20260808 10:06:29.476037 16448 leader_election.cc:288] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:06:29.476075 16456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:06:29.476194 16456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [term 3 LEADER]: Becoming Leader. State: Replica: 1022331640fa4a76b6d0e8bcee763466, State: Running, Role: LEADER
I20260808 10:06:29.476315 16448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:29.476284 16456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } }
I20260808 10:06:29.476585 16457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:06:29.476595 16459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1022331640fa4a76b6d0e8bcee763466. Latest consensus state: current_term: 3 leader_uuid: "1022331640fa4a76b6d0e8bcee763466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1022331640fa4a76b6d0e8bcee763466" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 34941 } } }
I20260808 10:06:29.476753 16457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:29.476770 16459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:29.477295 16464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:29.478152 16464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:29.478266 16464 catalog_manager.cc:1259] Loaded cluster ID: 6343652fa1c64d98b9ae47f44a0c2b43
I20260808 10:06:29.478308 16464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:29.479497 16464 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:29.479687 16464 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Loaded TSK: 0
I20260808 10:06:29.479740 16464 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1022331640fa4a76b6d0e8bcee763466: Loaded TSK: 1
F20260808 10:06:29.479959 16464 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc16dc2d9ad google::LogMessage::Fail() at ??:0
@ 0x7fc16dc3088a google::LogMessage::SendToLog() at ??:0
@ 0x7fc16dc2d4de google::LogMessage::Flush() at ??:0
@ 0x7fc16dc2ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc17039a104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc170397d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc175b8b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc175b8b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc175b8b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc175b51c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc175b5bad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc16fafb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc16faf0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc1709546db start_thread at ??:0
@ 0x7fc16c72b61f clone at ??:0
W20260808 10:06:29.577631 16454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:29.577917 16454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:29.577955 16454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:29.579710 16454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:29.579783 16454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:29.579823 16454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:29.582052 16454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:33989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=36183
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:29.582316 16454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:29.582604 16454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:29.588140 16501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:29.588313 16454 server_base.cc:1034] running on GCE node
W20260808 10:06:29.588429 16500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.588436 16504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:29.588712 16454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:29.588964 16454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:29.590122 16454 hybrid_clock.cc:648] HybridClock initialized: now 1786183589590102 us; error 41 us; skew 500 ppm
Aug 08 10:06:29 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:29.592643 16454 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:06:29.593067 16454 webserver.cc:460] Webserver started at http://127.24.162.129:36183/ using document root <none> and password file <none>
I20260808 10:06:29.593292 16454 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:29.593364 16454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:29.594833 16454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:29.595628 16514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.595825 16454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:29.595928 16454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "faf82053fb4848dd9ce81f3687c6dc79"
format_stamp: "Formatted at 2026-08-08 10:05:47 on dist-test-slave-nfb5"
server_key: "de97950d1026662fb725ad54de7fafd0"
server_key_iv: "4be8751ef6aecf2c111b723333fb1a2c"
server_key_version: "encryptionkey@0"
I20260808 10:06:29.596274 16454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:29.604553 16454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:29.606791 16454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:29.606992 16454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:29.607226 16454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:29.607681 16525 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:06:29.609719 16454 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:06:29.609778 16454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:06:29.609827 16454 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:06:29.610494 16454 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:06:29.610555 16454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:06:29.610639 16525 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap starting.
I20260808 10:06:29.617949 16454 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:33989
I20260808 10:06:29.617993 16682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:33989 every 8 connection(s)
I20260808 10:06:29.618376 16454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:29.620832 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 16454
I20260808 10:06:29.621163 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:46659
--local_ip_for_outbound_sockets=127.24.162.130
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=42079
--webserver_interface=127.24.162.130
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:06:29.621791 16525 log.cc:824] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:29.624643 16525 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:06:29.625008 16525 tablet_bootstrap.cc:492] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Bootstrap complete.
I20260808 10:06:29.626282 16525 ts_tablet_manager.cc:1397] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260808 10:06:29.627841 16525 raft_consensus.cc:348] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:29.628173 16525 raft_consensus.cc:729] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf82053fb4848dd9ce81f3687c6dc79, State: Initialized, Role: FOLLOWER
I20260808 10:06:29.628356 16525 consensus_queue.cc:260] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:29.628647 16525 ts_tablet_manager.cc:1428] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79: Time spent starting tablet: real 0.002s user 0.000s sys 0.006s
W20260808 10:06:29.703413 16686 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:06:29.619145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:35611)
0808 10:06:29.619319 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.619364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:29.619823 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:29.703210 (+ 83387us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
W20260808 10:06:29.703694 16683 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:34941 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:34941: Client connection negotiation failed: client connection to 127.24.162.190:34941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:06:29.704085 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.703942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:58031)
0808 10:06:29.704001 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.704054 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:06:29.704470 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.704366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:38907)
0808 10:06:29.704419 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.704449 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 10:06:29.704603 16683 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:06:29.708524 16689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:29.708705 16689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:29.708729 16689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:29.709878 16689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:29.709920 16689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:29.709973 16689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:29.711529 16689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:46659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=42079
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:29.711779 16689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:29.712026 16689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:29.717093 16696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.717087 16699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.717089 16697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:29.717105 16689 server_base.cc:1034] running on GCE node
I20260808 10:06:29.717489 16689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:29.717679 16689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:29.718816 16689 hybrid_clock.cc:648] HybridClock initialized: now 1786183589718815 us; error 41 us; skew 500 ppm
Aug 08 10:06:29 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:29.721405 16689 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:06:29.721863 16689 webserver.cc:460] Webserver started at http://127.24.162.130:42079/ using document root <none> and password file <none>
I20260808 10:06:29.722115 16689 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:29.722188 16689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:29.723585 16689 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:06:29.724189 16709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.724361 16689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:29.724468 16689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "dc16de8f52e1493bb52690ae2f96840c"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "ff56ed8f01aeb0e5aa9c5331e31dc2bb"
server_key_iv: "8f1aec306cb59cee19dbdbf5ee211f87"
server_key_version: "encryptionkey@0"
I20260808 10:06:29.724797 16689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:29.746156 16689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:29.748430 16689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:29.748597 16689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:29.748847 16689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:29.749176 16689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:29.749228 16689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.749274 16689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:29.749305 16689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.755590 16689 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:46659
I20260808 10:06:29.755724 16866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:46659 every 8 connection(s)
I20260808 10:06:29.756047 16689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:29.757205 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.756762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:54761)
0808 10:06:29.756976 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.757058 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
W20260808 10:06:29.757629 16868 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:34941 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:34941: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
I20260808 10:06:29.757934 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.757828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:58495)
0808 10:06:29.757879 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.757906 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:06:29.758209 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.758122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:60051)
0808 10:06:29.758165 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.758189 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:06:29.758318 16868 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:06:29.759932 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 16689
I20260808 10:06:29.760272 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:33575
--local_ip_for_outbound_sockets=127.24.162.131
--tserver_master_addrs=127.24.162.190:34941
--webserver_port=45305
--webserver_interface=127.24.162.131
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:29.843331 16875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:29.843565 16875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:29.843605 16875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:29.844918 16875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:29.844983 16875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:29.845026 16875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:29.846728 16875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:33575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=45305
--tserver_master_addrs=127.24.162.190:34941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:29.847046 16875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:29.847289 16875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:29.852376 16880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.852376 16887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:29.852398 16883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:29.852487 16875 server_base.cc:1034] running on GCE node
I20260808 10:06:29.852809 16875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:29.853072 16875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:29.854211 16875 hybrid_clock.cc:648] HybridClock initialized: now 1786183589854201 us; error 32 us; skew 500 ppm
Aug 08 10:06:29 dist-test-slave-nfb5 krb5kdc[14855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:29.856833 16875 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:06:29.857293 16875 webserver.cc:460] Webserver started at http://127.24.162.131:45305/ using document root <none> and password file <none>
I20260808 10:06:29.857514 16875 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:29.857560 16875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:29.858888 16875 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:29.859601 16899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.859808 16875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:06:29.859891 16875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "6c713adb186542f5988ce1e9227fc6f2"
format_stamp: "Formatted at 2026-08-08 10:05:48 on dist-test-slave-nfb5"
server_key: "340ad8ac600c60da8ad986dcdf9d5a2b"
server_key_iv: "640f6f49dbd90239492425f46929abed"
server_key_version: "encryptionkey@0"
I20260808 10:06:29.860231 16875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:29.871170 16875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:29.872999 16875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:29.873152 16875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:29.873379 16875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:29.873682 16875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:29.873715 16875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.873737 16875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:29.873780 16875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:29.880045 16875 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:33575
I20260808 10:06:29.880100 17056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:33575 every 8 connection(s)
I20260808 10:06:29.880442 16875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:29.881520 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.881145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:51479)
0808 10:06:29.881298 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.881374 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260808 10:06:29.881949 17059 heartbeater.cc:644] Failed to heartbeat to 127.24.162.190:34941 (0 consecutive failures): Network error: Failed to ping master at 127.24.162.190:34941: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
I20260808 10:06:29.882256 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.882159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:56473)
0808 10:06:29.882209 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.882238 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:06:29.882548 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:29.882474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:39435)
0808 10:06:29.882512 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:29.882534 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 10:06:29.882678 17059 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:06:29.886025 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 16875
I20260808 10:06:30.705340 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:30.705055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:55581)
0808 10:06:30.705214 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:30.705255 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260808 10:06:30.759003 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:30.758675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:40517)
0808 10:06:30.758824 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:30.758874 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260808 10:06:30.883412 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:30.883175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:33061)
0808 10:06:30.883323 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:30.883361 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260808 10:06:31.068125 17071 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:31.068267 17071 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:31.068645 17071 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
Aug 08 10:06:31 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.131@KRBTEST.COM
Aug 08 10:06:31 dist-test-slave-nfb5 krb5kdc[14855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183589, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.130@KRBTEST.COM
I20260808 10:06:31.088229 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.069016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:46659 (local address 127.24.162.129:47663)
0808 10:06:31.069135 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:31.069146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:31.069192 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:31.070037 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:31.070038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:31.070443 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:31.070622 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:31.070627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.071787 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.071788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:31.072599 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:31.072601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.073470 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.073473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:31.073535 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:31.082087 (+ 8552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:31.082093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:31.084279 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:31.086523 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:31.086524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:31.086525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:31.086724 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:31.087055 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:31.087055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:31.087056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:31.087124 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:31.087578 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:31.087587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:31.087754 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 10:06:31.087879 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:06:31.088082 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":11,"thread_start_us":71,"threads_started":1}
I20260808 10:06:31.088248 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.068957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:40559 (local address 127.24.162.131:33575)
0808 10:06:31.069145 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:31.069161 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:31.069223 (+ 62us) server_negotiation.cc:407] Connection header received
0808 10:06:31.069708 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:31.069711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:31.069757 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:31.069990 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:31.070762 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.071676 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.072704 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.073114 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.073254 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:31.083992 (+ 10738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:31.083998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:31.083999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:31.084014 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:31.086196 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:31.086647 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:31.086650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:31.086650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:31.086745 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:31.087095 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:31.087097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:31.087099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:31.087493 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:31.087596 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:31.087968 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 10:06:31.088122 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260808 10:06:31.088228 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.068844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:33575 (local address 127.24.162.129:40559)
0808 10:06:31.068920 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:31.068935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:31.069011 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:31.070034 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:31.070037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:31.070441 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:31.070622 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:31.070627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.071783 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.071784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:31.072598 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:31.072601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.073228 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.073229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:31.073341 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:31.082077 (+ 8736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:31.082087 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:31.083831 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:31.086286 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:31.086292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:31.086294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:31.086552 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:31.086842 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:31.086843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:31.086843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:31.087004 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:31.087578 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:31.087587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:31.087754 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 10:06:31.087878 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:06:31.088137 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":201}
I20260808 10:06:31.088248 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.069108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:47663 (local address 127.24.162.130:46659)
0808 10:06:31.069252 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:31.069263 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:31.069298 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:06:31.069728 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:31.069730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:31.069858 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:31.070009 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:31.070759 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.071679 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.072694 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:31.073106 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:31.073253 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:31.084397 (+ 11144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:31.084401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:31.084402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:31.084421 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:31.086444 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:31.086808 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:31.086811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:31.086811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:31.086980 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:31.087193 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:31.087193 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:31.087194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:31.087493 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:31.087596 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:31.087968 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 10:06:31.088122 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
W20260808 10:06:31.089561 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:31.089635 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:31.089653 16518 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faf82053fb4848dd9ce81f3687c6dc79; no voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c
I20260808 10:06:31.089811 17071 raft_consensus.cc:2726] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:06:31.706054 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.705849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:58817)
0808 10:06:31.705973 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:31.706005 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260808 10:06:31.759707 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.759498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:49049)
0808 10:06:31.759624 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:31.759665 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260808 10:06:31.884094 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:31.883885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:52255)
0808 10:06:31.884012 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:31.884051 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260808 10:06:32.706773 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:32.706527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:32849)
0808 10:06:32.706681 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:32.706721 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260808 10:06:32.760397 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:32.760192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:33293)
0808 10:06:32.760315 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:32.760356 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260808 10:06:32.884807 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:32.884563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:36281)
0808 10:06:32.884725 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:32.884766 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
I20260808 10:06:32.954471 17086 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:32.954573 17086 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:32.954811 17086 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
W20260808 10:06:32.955295 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:32.955384 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:32.955432 16518 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faf82053fb4848dd9ce81f3687c6dc79; no voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c
I20260808 10:06:32.955524 17086 raft_consensus.cc:2726] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:06:33.707584 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:33.707344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:60329)
0808 10:06:33.707489 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:33.707533 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260808 10:06:33.761175 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:33.760940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:57091)
0808 10:06:33.761084 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:33.761129 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260808 10:06:33.885623 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:33.885402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:54079)
0808 10:06:33.885540 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:33.885580 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260808 10:06:34.708431 17093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:34.708170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:39899)
0808 10:06:34.708343 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:34.708384 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260808 10:06:34.753845 17095 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:34.753954 17095 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:34.754163 17095 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
W20260808 10:06:34.754771 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:34.754863 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:34.754901 16518 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faf82053fb4848dd9ce81f3687c6dc79; no voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c
I20260808 10:06:34.755015 17095 raft_consensus.cc:2726] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:06:34.761861 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:34.761666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:56797)
0808 10:06:34.761784 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:34.761823 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260808 10:06:34.886348 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:34.886117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:36949)
0808 10:06:34.886262 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:34.886302 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260808 10:06:35.709118 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:35.708890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:54785)
0808 10:06:35.709032 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:35.709069 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260808 10:06:35.762557 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:35.762346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:37363)
0808 10:06:35.762471 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:35.762512 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260808 10:06:35.887072 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:35.886823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:53895)
0808 10:06:35.886995 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:35.887029 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260808 10:06:36.709941 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:36.709701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:42727)
0808 10:06:36.709857 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:36.709892 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260808 10:06:36.763305 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:36.763070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:53807)
0808 10:06:36.763218 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:36.763258 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260808 10:06:36.887771 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:36.887543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:54887)
0808 10:06:36.887688 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:36.887725 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260808 10:06:37.282889 17107 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:37.283064 17107 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:37.283244 17107 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
W20260808 10:06:37.283674 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:37.283771 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:37.283802 16518 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faf82053fb4848dd9ce81f3687c6dc79; no voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c
I20260808 10:06:37.283892 17107 raft_consensus.cc:2726] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:06:37.710695 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:37.710470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:52297)
0808 10:06:37.710604 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:37.710644 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260808 10:06:37.764039 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:37.763804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:41839)
0808 10:06:37.763952 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:37.763996 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260808 10:06:37.888530 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:37.888298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:59757)
0808 10:06:37.888442 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:37.888483 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260808 10:06:38.711469 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:38.711254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:39691)
0808 10:06:38.711386 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:38.711421 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 10:06:38.764778 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:38.764532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:45805)
0808 10:06:38.764677 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:38.764731 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260808 10:06:38.819165 17114 raft_consensus.cc:482] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:38.819260 17114 raft_consensus.cc:504] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c713adb186542f5988ce1e9227fc6f2" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 33575 } } peers { permanent_uuid: "dc16de8f52e1493bb52690ae2f96840c" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 46659 } } peers { permanent_uuid: "faf82053fb4848dd9ce81f3687c6dc79" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 33989 } }
I20260808 10:06:38.819460 17114 leader_election.cc:288] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575), dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659)
W20260808 10:06:38.819819 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc16de8f52e1493bb52690ae2f96840c (127.24.162.130:46659): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
W20260808 10:06:38.819904 16518 leader_election.cc:341] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c713adb186542f5988ce1e9227fc6f2 (127.24.162.131:33575): Not found: Tablet not found: 19f47b96221e4d96aa6f623d90f93d38
I20260808 10:06:38.819937 16518 leader_election.cc:302] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faf82053fb4848dd9ce81f3687c6dc79; no voters: 6c713adb186542f5988ce1e9227fc6f2, dc16de8f52e1493bb52690ae2f96840c
I20260808 10:06:38.820035 17114 raft_consensus.cc:2726] T 19f47b96221e4d96aa6f623d90f93d38 P faf82053fb4848dd9ce81f3687c6dc79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:06:38.889293 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:38.889091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:50891)
0808 10:06:38.889213 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:38.889248 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260808 10:06:39.712261 17116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:39.711981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:54243)
0808 10:06:39.712157 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:39.712199 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260808 10:06:39.765712 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:39.765416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:44805)
0808 10:06:39.765582 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:39.765625 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260808 10:06:39.890000 17118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:39.889772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:38745)
0808 10:06:39.889916 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:39.889955 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260808 10:06:40.713025 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:40.712802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:60625)
0808 10:06:40.712946 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:40.712979 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260808 10:06:40.766475 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:40.766254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:54265)
0808 10:06:40.766390 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:40.766427 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260808 10:06:40.890718 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:40.890508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:53687)
0808 10:06:40.890634 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:40.890673 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260808 10:06:41.713740 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:41.713461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:52469)
0808 10:06:41.713603 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:41.713639 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260808 10:06:41.767292 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:41.767053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:48081)
0808 10:06:41.767205 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:41.767244 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260808 10:06:41.891358 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:41.891144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:33047)
0808 10:06:41.891274 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:41.891313 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260808 10:06:42.714658 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:42.714414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:33069)
0808 10:06:42.714568 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:42.714608 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260808 10:06:42.768064 17131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:42.767792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:37037)
0808 10:06:42.767968 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:42.768016 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260808 10:06:42.892129 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:42.891867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:52551)
0808 10:06:42.892034 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:42.892075 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260808 10:06:43.715449 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:43.715192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:52223)
0808 10:06:43.715359 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:43.715399 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":64,"threads_started":1}
I20260808 10:06:43.768832 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:43.768596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:36545)
0808 10:06:43.768744 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:43.768783 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260808 10:06:43.892827 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:43.892583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:34479)
0808 10:06:43.892745 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:43.892780 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
I20260808 10:06:44.716243 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:44.716005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.129:42793)
0808 10:06:44.716155 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:44.716193 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260808 10:06:44.769670 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:44.769428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.130:42119)
0808 10:06:44.769579 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:44.769623 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260808 10:06:44.887100 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 16454
I20260808 10:06:44.893595 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:44.893384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:34941 (local address 127.24.162.131:48071)
0808 10:06:44.893518 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:44.893552 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.190:34941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260808 10:06:44.897220 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 16689
I20260808 10:06:44.907299 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 16875
2026-08-08T10:06:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58605 ms)
[----------] 30 tests from SecurityITest (238195 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:44 dist-test-slave-nfb5 krb5kdc[17145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:44 dist-test-slave-nfb5 krb5kdc[17145](info): set up 2 sockets
Aug 08 10:06:44 dist-test-slave-nfb5 krb5kdc[17145](info): commencing operation
krb5kdc: starting...
W20260808 10:06:46.227695 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:06:46.355947 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33211
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:46.440169 17163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:46.440414 17163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:46.440452 17163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:46.441674 17163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:46.441756 17163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:46.441780 17163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:46.441811 17163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:46.441843 17163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:46.443298 17163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:46.443572 17163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:46.443805 17163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:06:46.448845 17163 server_base.cc:1034] running on GCE node
W20260808 10:06:46.448822 17171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.448865 17169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.448829 17174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:46.449359 17163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:46.449679 17163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:46.450958 17163 hybrid_clock.cc:648] HybridClock initialized: now 1786183606450902 us; error 28 us; skew 500 ppm
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:46.453366 17163 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:06:46.453837 17163 webserver.cc:460] Webserver started at http://127.24.162.190:42871/ using document root <none> and password file <none>
I20260808 10:06:46.454020 17163 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:46.454058 17163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:46.454149 17163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:46.455262 17163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "079e571e4204416887de3ec2e93f48d2"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd0afd37ac70b3399c89c87a2ff82ed1"
server_key_iv: "3bfbe2635604f824ea754b5f664dabd8"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.455565 17163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "079e571e4204416887de3ec2e93f48d2"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd0afd37ac70b3399c89c87a2ff82ed1"
server_key_iv: "3bfbe2635604f824ea754b5f664dabd8"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.456866 17163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:06:46.457607 17186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.457799 17163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:46.457876 17163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "079e571e4204416887de3ec2e93f48d2"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd0afd37ac70b3399c89c87a2ff82ed1"
server_key_iv: "3bfbe2635604f824ea754b5f664dabd8"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.457929 17163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:46.464661 17163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:46.466578 17163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:46.466722 17163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:46.470631 17163 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33211
I20260808 10:06:46.470705 17256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33211 every 8 connection(s)
I20260808 10:06:46.471139 17163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:46.472105 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:46.474516 17257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: Bootstrap starting.
I20260808 10:06:46.475194 17257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:46.475495 17257 log.cc:824] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:46.476217 17257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: No bootstrap required, opened a new log
I20260808 10:06:46.477867 17257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } }
I20260808 10:06:46.478003 17257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:46.478025 17257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079e571e4204416887de3ec2e93f48d2, State: Initialized, Role: FOLLOWER
I20260808 10:06:46.478197 17257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } }
I20260808 10:06:46.478259 17257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:46.478281 17257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:46.478350 17257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:46.479153 17257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } }
I20260808 10:06:46.479310 17257 leader_election.cc:302] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 079e571e4204416887de3ec2e93f48d2; no voters:
I20260808 10:06:46.479497 17257 leader_election.cc:288] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:46.479542 17262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:46.479727 17262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [term 1 LEADER]: Becoming Leader. State: Replica: 079e571e4204416887de3ec2e93f48d2, State: Running, Role: LEADER
I20260808 10:06:46.479784 17257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:46.479869 17262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } }
I20260808 10:06:46.480528 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "079e571e4204416887de3ec2e93f48d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } } }
I20260808 10:06:46.480649 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:46.480693 17266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 079e571e4204416887de3ec2e93f48d2. Latest consensus state: current_term: 1 leader_uuid: "079e571e4204416887de3ec2e93f48d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "079e571e4204416887de3ec2e93f48d2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33211 } } }
I20260808 10:06:46.480777 17266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:46.480912 17276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:46.481367 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 17163
I20260808 10:06:46.481499 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:46.481539 17276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:46.481732 25226 external_mini_cluster.cc:1428] Setting key 9720d71d865a9913b6a3e25005d204fb
I20260808 10:06:46.483759 17276 catalog_manager.cc:1347] Generated new cluster ID: 67df13bad81f4350bc17098d49a472ea
I20260808 10:06:46.483819 17276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:46.494201 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.482426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52236 (local address 127.24.162.190:33211)
0808 10:06:46.482597 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:46.482608 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:46.482832 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:06:46.482983 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:46.482991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:46.483062 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:46.483230 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:46.484065 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.485507 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.486583 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.487273 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.487385 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.490327 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:46.490332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:46.490332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:46.490345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:46.492408 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.492838 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.492841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.492841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.492959 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.493202 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.493202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.493203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.493532 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:46.493674 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:46.493804 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 10:06:46.493939 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260808 10:06:46.501330 17276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:46.502063 17276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:46.515075 17276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: Generated new TSK 0
I20260808 10:06:46.515307 17276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:06:46.546273 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:46.629673 17300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:46.629918 17300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:46.629958 17300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:46.631153 17300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:46.631215 17300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:46.631259 17300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:46.632854 17300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:46.633121 17300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:46.633358 17300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:46.638475 17309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.638510 17307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.638492 17306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:46.638690 17300 server_base.cc:1034] running on GCE node
I20260808 10:06:46.638967 17300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:46.639236 17300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:46.640400 17300 hybrid_clock.cc:648] HybridClock initialized: now 1786183606640389 us; error 28 us; skew 500 ppm
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:46.643146 17300 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:06:46.643592 17300 webserver.cc:460] Webserver started at http://127.24.162.129:42329/ using document root <none> and password file <none>
I20260808 10:06:46.643822 17300 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:46.643893 17300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:46.644053 17300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:46.645105 17300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "64a2260baf3c4c699f68f9e1df2d0a74"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "e025647c469eba6c71cb01d1736c2c8f"
server_key_iv: "6fa96d9456ee10a6299b5e9fb93d7662"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.645488 17300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "64a2260baf3c4c699f68f9e1df2d0a74"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "e025647c469eba6c71cb01d1736c2c8f"
server_key_iv: "6fa96d9456ee10a6299b5e9fb93d7662"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.646828 17300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:46.647594 17320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.647766 17300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:46.647867 17300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "64a2260baf3c4c699f68f9e1df2d0a74"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "e025647c469eba6c71cb01d1736c2c8f"
server_key_iv: "6fa96d9456ee10a6299b5e9fb93d7662"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.647951 17300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:46.655354 17300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:46.657106 17300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:46.657274 17300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:46.657500 17300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:46.657801 17300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:46.657855 17300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.657896 17300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:46.657925 17300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.663901 17300 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:40535
I20260808 10:06:46.664011 17501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:40535 every 8 connection(s)
I20260808 10:06:46.664304 17300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:46.672254 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 17300
I20260808 10:06:46.672381 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:46.672610 25226 external_mini_cluster.cc:1428] Setting key ca0f4e566cb490465be12bfb594606a5
I20260808 10:06:46.674561 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.665012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33211 (local address 127.24.162.129:59061)
0808 10:06:46.665183 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:46.665249 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:46.665807 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:46.666018 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:46.666021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:46.666602 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:46.666863 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.666869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.667971 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.667972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.668794 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.668796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.669150 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.669151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.669269 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.669743 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:46.669751 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:46.671246 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:46.673095 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.673104 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.673106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.673439 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.673658 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.673659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.673659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.673760 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.674061 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:46.674067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:46.674184 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 10:06:46.674270 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:06:46.674392 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260808 10:06:46.674561 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.665077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:59061 (local address 127.24.162.190:33211)
0808 10:06:46.665161 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:46.665164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:46.665359 (+ 195us) server_negotiation.cc:407] Connection header received
0808 10:06:46.665900 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:46.665901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:46.665911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:46.665966 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:46.666991 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.667859 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.668875 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.669077 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.669128 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.671342 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:46.671343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:46.671343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:46.671351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:46.673021 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.673515 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.673517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.673517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.673589 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.673838 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.673839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.673839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.673991 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:46.674068 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:46.674319 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:06:46.674424 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:06:46.675765 17502 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33211
I20260808 10:06:46.675873 17502 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:46.676082 17502 heartbeater.cc:511] Master 127.24.162.190:33211 requested a full tablet report, sending...
I20260808 10:06:46.676654 17209 ts_manager.cc:194] Registered new tserver with Master: 64a2260baf3c4c699f68f9e1df2d0a74 (127.24.162.129:40535)
I20260808 10:06:46.677292 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:59061
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:06:46.701810 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:46.785542 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:46.785753 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:46.785777 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:46.786907 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:46.787002 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:46.787047 17513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:46.788497 17513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:46.788749 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:46.788990 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:46.795327 17521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.795343 17523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.795354 17526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:46.795789 17513 server_base.cc:1034] running on GCE node
I20260808 10:06:46.796092 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:46.796358 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:46.797506 17513 hybrid_clock.cc:648] HybridClock initialized: now 1786183606797489 us; error 35 us; skew 500 ppm
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:46.800133 17513 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:06:46.800537 17513 webserver.cc:460] Webserver started at http://127.24.162.130:38483/ using document root <none> and password file <none>
I20260808 10:06:46.800765 17513 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:46.800807 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:46.800896 17513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:46.801862 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "036b5f7e42ff45fa933ba22384d39208"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "c4c8ad28539912f3fdb44ac487a246a6"
server_key_iv: "72a7d8800b6c94ad6528d451f9a8d7cc"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.802161 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "036b5f7e42ff45fa933ba22384d39208"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "c4c8ad28539912f3fdb44ac487a246a6"
server_key_iv: "72a7d8800b6c94ad6528d451f9a8d7cc"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.803512 17513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:06:46.804203 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.804399 17513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:46.804486 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "036b5f7e42ff45fa933ba22384d39208"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "c4c8ad28539912f3fdb44ac487a246a6"
server_key_iv: "72a7d8800b6c94ad6528d451f9a8d7cc"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.804539 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:46.816215 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:46.818157 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:46.818270 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:46.818457 17513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:46.818706 17513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:46.818737 17513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.818763 17513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:46.818775 17513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.825459 17513 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:43411
I20260808 10:06:46.825496 17713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:43411 every 8 connection(s)
I20260808 10:06:46.825927 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:46.827234 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 17513
I20260808 10:06:46.827363 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:46.827580 25226 external_mini_cluster.cc:1428] Setting key eee2870279b338d9d79e60eead886c8c
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:46.837401 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.826772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:37995 (local address 127.24.162.190:33211)
0808 10:06:46.826844 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:46.826846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:46.827150 (+ 304us) server_negotiation.cc:407] Connection header received
0808 10:06:46.827801 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:46.827802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:46.827816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:46.827874 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:46.828934 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.829446 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.830724 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.831027 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.831095 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.833975 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:46.833976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:46.833977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:46.833984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:46.835567 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.836165 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.836166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.836166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.836220 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.836565 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.836566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.836567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.836755 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:46.836831 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:46.837255 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 10:06:46.837333 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:06:46.837700 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.826750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33211 (local address 127.24.162.130:37995)
0808 10:06:46.826996 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:46.827059 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:46.827689 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:46.827945 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:46.827949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:46.828623 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:46.828818 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.828822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.829542 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.829543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.830615 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.830617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.831116 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.831117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.831282 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.831922 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:46.831930 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:46.833874 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:46.835708 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.835716 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.835719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.836080 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.836293 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.836294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.836294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.836458 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.836848 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:46.836857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:46.837051 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 10:06:46.837207 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 10:06:46.837411 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
I20260808 10:06:46.838413 17714 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33211
I20260808 10:06:46.838557 17714 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:46.838822 17714 heartbeater.cc:511] Master 127.24.162.190:33211 requested a full tablet report, sending...
I20260808 10:06:46.839357 17209 ts_manager.cc:194] Registered new tserver with Master: 036b5f7e42ff45fa933ba22384d39208 (127.24.162.130:43411)
I20260808 10:06:46.839870 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:37995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:06:46.858703 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:46.940763 17720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:46.941025 17720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:46.941063 17720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:46.942207 17720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:46.942283 17720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:46.942325 17720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:46.943768 17720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:46.944046 17720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:46.944283 17720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:46.949223 17732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:46.949270 17731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:46.949432 17720 server_base.cc:1034] running on GCE node
W20260808 10:06:46.949270 17734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:46.949774 17720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:46.950039 17720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:46.951207 17720 hybrid_clock.cc:648] HybridClock initialized: now 1786183606951190 us; error 48 us; skew 500 ppm
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:46.953799 17720 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:06:46.954237 17720 webserver.cc:460] Webserver started at http://127.24.162.131:36343/ using document root <none> and password file <none>
I20260808 10:06:46.954460 17720 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:46.954531 17720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:46.954689 17720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:46.955811 17720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "f94f35fa5cf14e5c8533c4100912527b"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd4cc0a05060170fb4f4d7c2cc3281d4"
server_key_iv: "8d09abc3944d4a9618f0c41ae50dea7c"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.956192 17720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "f94f35fa5cf14e5c8533c4100912527b"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd4cc0a05060170fb4f4d7c2cc3281d4"
server_key_iv: "8d09abc3944d4a9618f0c41ae50dea7c"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.957619 17720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:06:46.958370 17745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.958549 17720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:46.958652 17720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "f94f35fa5cf14e5c8533c4100912527b"
format_stamp: "Formatted at 2026-08-08 10:06:46 on dist-test-slave-nfb5"
server_key: "bd4cc0a05060170fb4f4d7c2cc3281d4"
server_key_iv: "8d09abc3944d4a9618f0c41ae50dea7c"
server_key_version: "encryptionkey@0"
I20260808 10:06:46.958767 17720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:46.971254 17720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:46.973071 17720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:46.973234 17720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:46.973459 17720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:46.973771 17720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:46.973826 17720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.973867 17720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:46.973896 17720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:46.979905 17720 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:46275
I20260808 10:06:46.979975 17905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:46275 every 8 connection(s)
I20260808 10:06:46.980363 17720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:46.984269 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 17720
I20260808 10:06:46.984392 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:46.984601 25226 external_mini_cluster.cc:1428] Setting key 9766ea8a7a4a3d259edefde8e618abfe
Aug 08 10:06:46 dist-test-slave-nfb5 krb5kdc[17145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183606, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:46.990002 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.981185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:33391 (local address 127.24.162.190:33211)
0808 10:06:46.981233 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:46.981235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:46.981513 (+ 278us) server_negotiation.cc:407] Connection header received
0808 10:06:46.981970 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:46.981971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:46.981984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:46.982058 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:46.983091 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.983578 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.984665 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.984962 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.985080 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.987367 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:46.987368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:46.987368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:46.987375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:46.988800 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.989192 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.989193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.989193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.989243 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:46.989464 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:46.989465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:46.989465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:46.989570 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:46.989622 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:46.989877 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:06:46.989939 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:06:46.990207 17913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.981186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33211 (local address 127.24.162.131:33391)
0808 10:06:46.981356 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:46.981414 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:46.981877 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:46.982109 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:46.982112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:46.982745 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:46.982995 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.983001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.983665 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.983666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.984571 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:46.984574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.985116 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:46.985117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:46.985287 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:46.985865 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:46.985873 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:46.987278 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:46.988894 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.988900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.988902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.989125 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.989295 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:46.989296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:46.989296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:46.989409 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:46.989627 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:46.989632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:46.989754 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 10:06:46.989839 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:06:46.989956 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260808 10:06:46.990679 17908 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33211
I20260808 10:06:46.990795 17908 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:46.991041 17908 heartbeater.cc:511] Master 127.24.162.190:33211 requested a full tablet report, sending...
I20260808 10:06:46.991400 17209 ts_manager.cc:194] Registered new tserver with Master: f94f35fa5cf14e5c8533c4100912527b (127.24.162.131:46275)
I20260808 10:06:46.991770 17209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:33391
I20260808 10:06:46.996641 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:47.003958 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:46.997976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.24.162.190:33211)
0808 10:06:46.998048 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:46.998050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:46.998100 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:06:46.998135 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:46.998136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:46.998144 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:46.998190 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:46.998693 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:46.999211 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.000000 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.000175 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.000224 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.001394 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:47.001395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:47.001396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:47.001402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:47.002810 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:47.003199 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:47.003200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:47.003200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:47.003255 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:47.003440 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:47.003440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:47.003440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:47.003575 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:47.003651 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.003807 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.003866 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:06:47.005595 17209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:47.007303 17209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:47.013957 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.009829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:46275 (local address 127.0.0.1:40856)
0808 10:06:47.010358 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.010389 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.010485 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.010791 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.010792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.010794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:47.010861 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.010863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.011721 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.011722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.012861 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.012863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.013660 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.013661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.013750 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.013763 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.013837 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.013883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":81,"threads_started":1}
I20260808 10:06:47.014008 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.009968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.24.162.131:46275)
0808 10:06:47.010156 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.010162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.010459 (+ 297us) server_negotiation.cc:407] Connection header received
0808 10:06:47.010583 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.010586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.010604 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.010661 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:47.010973 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.011623 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.012948 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.013570 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.013645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.013698 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.013873 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.013915 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260808 10:06:47.014343 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.009823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:43411 (local address 127.0.0.1:54720)
0808 10:06:47.010344 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.010389 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.010466 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.010726 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.010730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.010732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:47.010806 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.010808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.012139 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.012140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.013130 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.013131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.014064 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.014065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.014142 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.014145 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.014227 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.014267 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":81,"threads_started":1}
I20260808 10:06:47.014355 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.010041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.24.162.130:43411)
0808 10:06:47.010207 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.010211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.010548 (+ 337us) server_negotiation.cc:407] Connection header received
0808 10:06:47.010592 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.010595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.010609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.010707 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:47.010893 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.012017 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.013205 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.013909 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.013978 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.014025 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.014243 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.014277 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260808 10:06:47.015180 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.009823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:40535 (local address 127.0.0.1:50170)
0808 10:06:47.010503 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.010513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.010561 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.010846 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.010847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.010848 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:47.010887 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.010888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.013207 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.013207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.014048 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.014051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.014950 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.014951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.015013 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.015018 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.015063 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.015105 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":350,"thread_start_us":86,"threads_started":1}
I20260808 10:06:47.015125 17814 tablet_service.cc:1467] Processing CreateTablet for tablet a4115e50321841a2ab2450ca0e3f719a (DEFAULT_TABLE table=test-table [id=da442a6ce62e4bc1a4f7741b54507061]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:47.015329 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.010196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.24.162.129:40535)
0808 10:06:47.010647 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.010672 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.010699 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:06:47.010737 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.010739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.010754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.010845 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:47.010983 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.012034 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.014152 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.014851 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.015006 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.015073 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.015097 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.015136 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":80,"threads_started":1}
I20260808 10:06:47.015359 17609 tablet_service.cc:1467] Processing CreateTablet for tablet a4115e50321841a2ab2450ca0e3f719a (DEFAULT_TABLE table=test-table [id=da442a6ce62e4bc1a4f7741b54507061]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:47.015542 17814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4115e50321841a2ab2450ca0e3f719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:47.015769 17609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4115e50321841a2ab2450ca0e3f719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:47.016129 17402 tablet_service.cc:1467] Processing CreateTablet for tablet a4115e50321841a2ab2450ca0e3f719a (DEFAULT_TABLE table=test-table [id=da442a6ce62e4bc1a4f7741b54507061]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:47.016441 17402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4115e50321841a2ab2450ca0e3f719a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:47.019080 17931 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Bootstrap starting.
I20260808 10:06:47.019635 17931 tablet_bootstrap.cc:654] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:47.019752 17933 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Bootstrap starting.
I20260808 10:06:47.019771 17932 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Bootstrap starting.
I20260808 10:06:47.020285 17931 log.cc:824] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:47.020399 17932 tablet_bootstrap.cc:654] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:47.020465 17933 tablet_bootstrap.cc:654] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:47.021003 17932 log.cc:824] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:47.021028 17933 log.cc:824] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:47.021288 17931 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: No bootstrap required, opened a new log
I20260808 10:06:47.021368 17931 ts_tablet_manager.cc:1397] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:47.021744 17933 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: No bootstrap required, opened a new log
I20260808 10:06:47.021813 17932 tablet_bootstrap.cc:492] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: No bootstrap required, opened a new log
I20260808 10:06:47.021831 17933 ts_tablet_manager.cc:1397] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:47.021878 17932 ts_tablet_manager.cc:1397] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:47.023209 17933 raft_consensus.cc:348] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.023377 17933 raft_consensus.cc:374] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:47.023424 17933 raft_consensus.cc:729] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a2260baf3c4c699f68f9e1df2d0a74, State: Initialized, Role: FOLLOWER
I20260808 10:06:47.023396 17931 raft_consensus.cc:348] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.023583 17931 raft_consensus.cc:374] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:47.023540 17933 consensus_queue.cc:260] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.023630 17931 raft_consensus.cc:729] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f94f35fa5cf14e5c8533c4100912527b, State: Initialized, Role: FOLLOWER
I20260808 10:06:47.023759 17931 consensus_queue.cc:260] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.023821 17933 ts_tablet_manager.cc:1428] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:47.023962 17502 heartbeater.cc:503] Master 127.24.162.190:33211 was elected leader, sending a full tablet report...
I20260808 10:06:47.023849 17932 raft_consensus.cc:348] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.024008 17932 raft_consensus.cc:374] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:47.024044 17931 ts_tablet_manager.cc:1428] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:47.024052 17932 raft_consensus.cc:729] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 036b5f7e42ff45fa933ba22384d39208, State: Initialized, Role: FOLLOWER
I20260808 10:06:47.024155 17932 consensus_queue.cc:260] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.024209 17908 heartbeater.cc:503] Master 127.24.162.190:33211 was elected leader, sending a full tablet report...
I20260808 10:06:47.024415 17932 ts_tablet_manager.cc:1428] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:06:47.024542 17714 heartbeater.cc:503] Master 127.24.162.190:33211 was elected leader, sending a full tablet report...
W20260808 10:06:47.076532 17715 tablet.cc:2369] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:06:47.165047 17508 tablet.cc:2369] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:06:47.231081 17911 tablet.cc:2369] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:47.372855 17937 raft_consensus.cc:482] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:47.372994 17937 raft_consensus.cc:504] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.373432 17937 leader_election.cc:288] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 036b5f7e42ff45fa933ba22384d39208 (127.24.162.130:43411), f94f35fa5cf14e5c8533c4100912527b (127.24.162.131:46275)
I20260808 10:06:47.377699 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.373735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:54403 (local address 127.24.162.131:46275)
0808 10:06:47.373798 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.373800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.373898 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:06:47.373935 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.373936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.373946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.374023 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:47.374270 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.375059 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.376743 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.377309 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.377387 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.377395 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.377588 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.377625 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:06:47.377699 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.373807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:53497 (local address 127.24.162.130:43411)
0808 10:06:47.373849 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.373851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.373887 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:06:47.373940 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.373941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.373950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.374023 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:47.374270 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.375048 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.376743 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.377292 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.377387 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.377395 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.377588 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.377625 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:06:47.377722 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.373646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:43411 (local address 127.24.162.129:53497)
0808 10:06:47.373804 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.373817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.373882 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.374049 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.374050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.374055 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:47.374170 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.374171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.375150 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.375151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.376563 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.376566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.377406 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.377406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.377492 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.377499 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.377564 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.377613 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":91,"thread_start_us":116,"threads_started":1}
I20260808 10:06:47.377992 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.373646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:46275 (local address 127.24.162.129:54403)
0808 10:06:47.373815 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.373825 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.373883 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.374049 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.374050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.374053 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:47.374165 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.374167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.375149 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.375151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.376564 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.376566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.377403 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.377405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.377492 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.377500 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.377564 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.377613 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":111,"thread_start_us":67,"threads_started":1}
I20260808 10:06:47.378019 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4115e50321841a2ab2450ca0e3f719a" candidate_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "036b5f7e42ff45fa933ba22384d39208" is_pre_election: true
I20260808 10:06:47.378191 17643 raft_consensus.cc:2454] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64a2260baf3c4c699f68f9e1df2d0a74 in term 0.
I20260808 10:06:47.378355 17842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4115e50321841a2ab2450ca0e3f719a" candidate_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f94f35fa5cf14e5c8533c4100912527b" is_pre_election: true
I20260808 10:06:47.378407 17323 leader_election.cc:302] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 036b5f7e42ff45fa933ba22384d39208, 64a2260baf3c4c699f68f9e1df2d0a74; no voters:
I20260808 10:06:47.378485 17842 raft_consensus.cc:2454] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64a2260baf3c4c699f68f9e1df2d0a74 in term 0.
I20260808 10:06:47.378580 17937 raft_consensus.cc:2781] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:47.378647 17937 raft_consensus.cc:482] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:47.378677 17937 raft_consensus.cc:3037] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:47.379443 17937 raft_consensus.cc:504] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.379594 17937 leader_election.cc:288] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [CANDIDATE]: Term 1 election: Requested vote from peers 036b5f7e42ff45fa933ba22384d39208 (127.24.162.130:43411), f94f35fa5cf14e5c8533c4100912527b (127.24.162.131:46275)
I20260808 10:06:47.379769 17643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4115e50321841a2ab2450ca0e3f719a" candidate_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "036b5f7e42ff45fa933ba22384d39208"
I20260808 10:06:47.379812 17842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4115e50321841a2ab2450ca0e3f719a" candidate_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f94f35fa5cf14e5c8533c4100912527b"
I20260808 10:06:47.379876 17643 raft_consensus.cc:3037] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:47.379882 17842 raft_consensus.cc:3037] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:47.380610 17643 raft_consensus.cc:2454] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64a2260baf3c4c699f68f9e1df2d0a74 in term 1.
I20260808 10:06:47.380801 17842 raft_consensus.cc:2454] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64a2260baf3c4c699f68f9e1df2d0a74 in term 1.
I20260808 10:06:47.380815 17323 leader_election.cc:302] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 036b5f7e42ff45fa933ba22384d39208, 64a2260baf3c4c699f68f9e1df2d0a74; no voters:
I20260808 10:06:47.380909 17937 raft_consensus.cc:2781] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:47.381099 17937 raft_consensus.cc:686] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 1 LEADER]: Becoming Leader. State: Replica: 64a2260baf3c4c699f68f9e1df2d0a74, State: Running, Role: LEADER
I20260808 10:06:47.381228 17937 consensus_queue.cc:237] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } }
I20260808 10:06:47.382124 17208 catalog_manager.cc:5495] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64a2260baf3c4c699f68f9e1df2d0a74 (127.24.162.129). New cstate: current_term: 1 leader_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64a2260baf3c4c699f68f9e1df2d0a74" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 40535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 } health_report { overall_health: UNKNOWN } } }
I20260808 10:06:47.453305 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.450719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.24.162.129:40535)
0808 10:06:47.450802 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.450804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.450815 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:47.450865 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.450866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.450889 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.450999 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:47.451117 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.451692 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.452463 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.452675 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.452761 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.452959 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:47.453069 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:47.453165 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.453197 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.453217 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:06:47.455286 17842 raft_consensus.cc:1261] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 1 FOLLOWER]: Refusing update from remote peer 64a2260baf3c4c699f68f9e1df2d0a74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:47.455307 17643 raft_consensus.cc:1261] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 1 FOLLOWER]: Refusing update from remote peer 64a2260baf3c4c699f68f9e1df2d0a74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:47.455701 17944 consensus_queue.cc:1035] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "036b5f7e42ff45fa933ba22384d39208" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 43411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:47.455856 17937 consensus_queue.cc:1035] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f94f35fa5cf14e5c8533c4100912527b" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:47.458098 17949 mvcc.cc:205] Tried to move back new op lower bound from 7316208056133693440 to 7316208055834148864. Current Snapshot: MvccSnapshot[applied={T|T < 7316208056133693440}]
I20260808 10:06:47.458808 17953 mvcc.cc:205] Tried to move back new op lower bound from 7316208056133693440 to 7316208055834148864. Current Snapshot: MvccSnapshot[applied={T|T < 7316208056133693440}]
I20260808 10:06:47.459002 17952 mvcc.cc:205] Tried to move back new op lower bound from 7316208056133693440 to 7316208055834148864. Current Snapshot: MvccSnapshot[applied={T|T < 7316208056133693440 or (T in {7316208056133693440})}]
I20260808 10:06:47.460419 17209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:47.460525 17209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:47.461333 17209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 079e571e4204416887de3ec2e93f48d2: Sending DeleteTablet for 3 replicas of tablet a4115e50321841a2ab2450ca0e3f719a
I20260808 10:06:47.461717 17402 tablet_service.cc:1514] Processing DeleteTablet for tablet a4115e50321841a2ab2450ca0e3f719a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:50170
I20260808 10:06:47.461717 17814 tablet_service.cc:1514] Processing DeleteTablet for tablet a4115e50321841a2ab2450ca0e3f719a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:40856
I20260808 10:06:47.461937 17963 tablet_replica.cc:330] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: stopping tablet replica
I20260808 10:06:47.461936 17609 tablet_service.cc:1514] Processing DeleteTablet for tablet a4115e50321841a2ab2450ca0e3f719a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:47 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:54720
I20260808 10:06:47.462040 17963 raft_consensus.cc:2229] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:47.462145 17963 raft_consensus.cc:2258] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:47.462172 17964 tablet_replica.cc:330] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: stopping tablet replica
I20260808 10:06:47.462349 17964 raft_consensus.cc:2229] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:47.462478 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 17300
I20260808 10:06:47.462587 17964 raft_consensus.cc:2258] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:47.462598 17963 ts_tablet_manager.cc:1905] T a4115e50321841a2ab2450ca0e3f719a P 64a2260baf3c4c699f68f9e1df2d0a74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:47.465469 17964 ts_tablet_manager.cc:1905] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:47.467247 17964 ts_tablet_manager.cc:1918] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:47.467326 17964 log.cc:1196] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/a4115e50321841a2ab2450ca0e3f719a
I20260808 10:06:47.467555 17964 ts_tablet_manager.cc:1939] T a4115e50321841a2ab2450ca0e3f719a P 036b5f7e42ff45fa933ba22384d39208: Deleting consensus metadata
I20260808 10:06:47.468279 17190 catalog_manager.cc:4841] TS 036b5f7e42ff45fa933ba22384d39208 (127.24.162.130:43411): tablet a4115e50321841a2ab2450ca0e3f719a (table test-table [id=da442a6ce62e4bc1a4f7741b54507061]) successfully deleted
I20260808 10:06:47.472687 17962 tablet_replica.cc:330] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: stopping tablet replica
I20260808 10:06:47.472792 17962 raft_consensus.cc:2229] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:47.472944 17962 raft_consensus.cc:2258] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:47.473423 17962 ts_tablet_manager.cc:1905] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:06:47.473912 17191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:40535 (error 108)
I20260808 10:06:47.474474 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.474314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:40535 (local address 127.0.0.1:35586)
0808 10:06:47.474382 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.474435 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:06:47.474543 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 17513
I20260808 10:06:47.476533 17962 ts_tablet_manager.cc:1918] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:47.476598 17962 log.cc:1196] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/a4115e50321841a2ab2450ca0e3f719a
I20260808 10:06:47.476848 17962 ts_tablet_manager.cc:1939] T a4115e50321841a2ab2450ca0e3f719a P f94f35fa5cf14e5c8533c4100912527b: Deleting consensus metadata
I20260808 10:06:47.477346 17189 catalog_manager.cc:4841] TS f94f35fa5cf14e5c8533c4100912527b (127.24.162.131:46275): tablet a4115e50321841a2ab2450ca0e3f719a (table test-table [id=da442a6ce62e4bc1a4f7741b54507061]) successfully deleted
W20260808 10:06:47.480368 17191 catalog_manager.cc:4584] TS 64a2260baf3c4c699f68f9e1df2d0a74 (127.24.162.129:40535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4115e50321841a2ab2450ca0e3f719a: Network error: Client connection negotiation failed: client connection to 127.24.162.129:40535: connect: Connection refused (error 111)
I20260808 10:06:47.484413 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 17720
I20260808 10:06:47.493737 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 17163
2026-08-08T10:06:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2588 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T10:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:47Z Disabled control of system clock
I20260808 10:06:47.529246 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36707
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:47.611341 17973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:47.611552 17973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:47.611589 17973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:47.612697 17973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:47.612733 17973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:47.612747 17973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:47.612759 17973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:47.612771 17973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:47.614148 17973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:47.614358 17973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:47.614593 17973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:47.619436 17982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:47.619534 17979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:47.619666 17973 server_base.cc:1034] running on GCE node
W20260808 10:06:47.619542 17978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:47.620014 17973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:47.620311 17973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:47.621449 17973 hybrid_clock.cc:648] HybridClock initialized: now 1786183607621436 us; error 27 us; skew 500 ppm
I20260808 10:06:47.622596 17973 webserver.cc:460] Webserver started at http://127.24.162.190:36105/ using document root <none> and password file <none>
I20260808 10:06:47.622814 17973 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:47.622856 17973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:47.623011 17973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:47.624056 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "20409bee0f904a829daf18c058d2fb28"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "8cfadfa8d1ea4875cf595aba08f58c04"
server_key_iv: "a5c348d129f02696a8a6a6adf139c97e"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.624361 17973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "20409bee0f904a829daf18c058d2fb28"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "8cfadfa8d1ea4875cf595aba08f58c04"
server_key_iv: "a5c348d129f02696a8a6a6adf139c97e"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.625749 17973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:47.626463 17992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.626727 17973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:47.626847 17973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "20409bee0f904a829daf18c058d2fb28"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "8cfadfa8d1ea4875cf595aba08f58c04"
server_key_iv: "a5c348d129f02696a8a6a6adf139c97e"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.626945 17973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:47.646507 17973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:47.646896 17973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:47.647125 17973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:47.651120 17973 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36707
I20260808 10:06:47.651181 18069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36707 every 8 connection(s)
I20260808 10:06:47.651525 17973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:47.652585 18070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:47.654091 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 17973
I20260808 10:06:47.654199 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:47.654409 25226 external_mini_cluster.cc:1428] Setting key a6d0f582fbc0625fe573709022dfa62e
I20260808 10:06:47.655594 18070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: Bootstrap starting.
I20260808 10:06:47.656306 18070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:47.656621 18070 log.cc:824] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:47.657367 18070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: No bootstrap required, opened a new log
I20260808 10:06:47.659857 18070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } }
I20260808 10:06:47.660032 18070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:47.660058 18070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20409bee0f904a829daf18c058d2fb28, State: Initialized, Role: FOLLOWER
I20260808 10:06:47.660198 18070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } }
I20260808 10:06:47.660271 18070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:47.660293 18070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:47.660372 18070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:47.660866 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.655052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.24.162.190:36707)
0808 10:06:47.655267 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.655274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.655299 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:06:47.655740 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.655747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.655819 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.655935 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:47.656244 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.657377 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.658532 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.659026 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.659118 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.659946 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:47.659950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:47.659950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:47.660221 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:47.660279 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:47.660295 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:47.660364 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.660581 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.660717 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260808 10:06:47.660912 18070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } }
I20260808 10:06:47.661026 18070 leader_election.cc:302] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20409bee0f904a829daf18c058d2fb28; no voters:
I20260808 10:06:47.661201 18070 leader_election.cc:288] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:47.661338 18077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:47.661461 18070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:47.661492 18077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [term 1 LEADER]: Becoming Leader. State: Replica: 20409bee0f904a829daf18c058d2fb28, State: Running, Role: LEADER
I20260808 10:06:47.661608 18077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } }
I20260808 10:06:47.661962 18077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20409bee0f904a829daf18c058d2fb28. Latest consensus state: current_term: 1 leader_uuid: "20409bee0f904a829daf18c058d2fb28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } } }
I20260808 10:06:47.662055 18077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:47.662256 18080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20409bee0f904a829daf18c058d2fb28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20409bee0f904a829daf18c058d2fb28" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36707 } } }
I20260808 10:06:47.662278 18092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:47.662331 18080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:47.662887 18092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:47.664508 18092 catalog_manager.cc:1347] Generated new cluster ID: 9f99efaa44804501906c1fda08fae50a
I20260808 10:06:47.664559 18092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:47.669034 18092 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:47.669754 18092 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:47.675846 18092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: Generated new TSK 0
I20260808 10:06:47.676024 18092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:47.681810 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--builtin_ntp_servers=127.24.162.148:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:47.767670 18105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:47.767881 18105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:47.767904 18105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:47.769066 18105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:47.769109 18105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:47.769133 18105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:47.770474 18105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:47.770745 18105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:47.771114 18105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:47.777035 18114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:47.777035 18112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:47.777262 18105 server_base.cc:1034] running on GCE node
W20260808 10:06:47.777026 18111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:47.777558 18105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:47.777858 18105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:47.778976 18105 hybrid_clock.cc:648] HybridClock initialized: now 1786183607778955 us; error 37 us; skew 500 ppm
I20260808 10:06:47.780287 18105 webserver.cc:460] Webserver started at http://127.24.162.129:41969/ using document root <none> and password file <none>
I20260808 10:06:47.780560 18105 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:47.780632 18105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:47.780820 18105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:47.781880 18105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "efaa1a530c47aa30e7db34be1e90184c"
server_key_iv: "74a0c0faaedf3b1b6e2c8212b6838855"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.782246 18105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "efaa1a530c47aa30e7db34be1e90184c"
server_key_iv: "74a0c0faaedf3b1b6e2c8212b6838855"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.783648 18105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:47.784425 18122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.784584 18105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:47.784696 18105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "efaa1a530c47aa30e7db34be1e90184c"
server_key_iv: "74a0c0faaedf3b1b6e2c8212b6838855"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.784778 18105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:47.812716 18105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:47.813064 18105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:47.813228 18105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:47.813478 18105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:47.813823 18105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:47.813858 18105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.813881 18105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:47.813930 18105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.820134 18105 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:42733
I20260808 10:06:47.820187 18297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:42733 every 8 connection(s)
I20260808 10:06:47.820520 18105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:47.825593 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.821284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:55979 (local address 127.24.162.190:36707)
0808 10:06:47.821362 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.821364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.821584 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:06:47.822069 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.822070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.822085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.822145 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:47.822803 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.823375 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.824386 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.824564 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.824615 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.825384 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:47.825385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:47.825386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:47.825390 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:47.825404 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:47.825408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:47.825450 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.825514 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.825535 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:06:47.825783 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.821265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36707 (local address 127.24.162.129:55979)
0808 10:06:47.821449 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.821491 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.821985 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.822219 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.822222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.822230 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:47.822727 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.822731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.823493 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.823494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.824302 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.824305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.824643 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.824644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.824767 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.825253 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:47.825253 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:47.825280 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:47.825301 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:47.825450 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:47.825452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:47.825454 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.825498 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.825570 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260808 10:06:47.826267 18298 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36707
I20260808 10:06:47.826354 18298 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:47.826551 18298 heartbeater.cc:511] Master 127.24.162.190:36707 requested a full tablet report, sending...
I20260808 10:06:47.827107 18016 ts_manager.cc:194] Registered new tserver with Master: a58e8fa6b2d24f868ee5c36c16eaf52a (127.24.162.129:42733)
I20260808 10:06:47.827731 18016 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.129:55979
I20260808 10:06:47.828374 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 18105
I20260808 10:06:47.828485 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:47.828693 25226 external_mini_cluster.cc:1428] Setting key c5803079266d801acdf11e9434ba3266
I20260808 10:06:47.829650 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--builtin_ntp_servers=127.24.162.148:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:47.913311 18304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:47.913525 18304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:47.913547 18304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:47.914700 18304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:47.914744 18304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:47.914769 18304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:47.916205 18304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:47.916456 18304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:47.916776 18304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:47.921921 18312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:47.921908 18311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:47.921975 18316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:47.922147 18304 server_base.cc:1034] running on GCE node
I20260808 10:06:47.922407 18304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:47.922653 18304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:47.923801 18304 hybrid_clock.cc:648] HybridClock initialized: now 1786183607923767 us; error 48 us; skew 500 ppm
I20260808 10:06:47.925067 18304 webserver.cc:460] Webserver started at http://127.24.162.130:42897/ using document root <none> and password file <none>
I20260808 10:06:47.925338 18304 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:47.925411 18304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:47.925561 18304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:47.926625 18304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "10417c8c58c04baf99ebafc075e648bf"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "484487a1849e1ab663c2dd69778081a8"
server_key_iv: "4efac5bc0f6a678ff4cb7353bfa149d8"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.927059 18304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "10417c8c58c04baf99ebafc075e648bf"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "484487a1849e1ab663c2dd69778081a8"
server_key_iv: "4efac5bc0f6a678ff4cb7353bfa149d8"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.928442 18304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:47.929150 18322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.929332 18304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:47.929431 18304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "10417c8c58c04baf99ebafc075e648bf"
format_stamp: "Formatted at 2026-08-08 10:06:47 on dist-test-slave-nfb5"
server_key: "484487a1849e1ab663c2dd69778081a8"
server_key_iv: "4efac5bc0f6a678ff4cb7353bfa149d8"
server_key_version: "encryptionkey@0"
I20260808 10:06:47.929513 18304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:47.948149 18304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:47.948457 18304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:47.948607 18304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:47.948835 18304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:47.949162 18304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:47.949219 18304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.949254 18304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:47.949280 18304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:47.955228 18304 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:40971
I20260808 10:06:47.955281 18494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:40971 every 8 connection(s)
I20260808 10:06:47.955605 18304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:47.960520 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.956384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:56067 (local address 127.24.162.190:36707)
0808 10:06:47.956433 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:47.956435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:47.956661 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:06:47.957117 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:47.957118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:47.957128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:47.957188 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:47.957826 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.958316 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.959300 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.959482 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.959538 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.960288 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:47.960290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:47.960290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:47.960294 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:47.960308 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:47.960312 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:47.960356 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:47.960438 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:06:47.960462 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:06:47.960709 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:47.956327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36707 (local address 127.24.162.130:56067)
0808 10:06:47.956520 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:47.956562 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:47.957035 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:47.957242 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:47.957246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:47.957254 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:47.957746 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.957750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.958405 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.958406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.959220 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:47.959222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:47.959557 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:47.959558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:47.959662 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:47.960140 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:47.960140 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:47.960167 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:47.960207 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:47.960368 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:47.960373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:47.960375 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:47.960427 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:06:47.960503 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260808 10:06:47.961288 18497 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36707
I20260808 10:06:47.961406 18497 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:47.961644 18497 heartbeater.cc:511] Master 127.24.162.190:36707 requested a full tablet report, sending...
I20260808 10:06:47.962034 18016 ts_manager.cc:194] Registered new tserver with Master: 10417c8c58c04baf99ebafc075e648bf (127.24.162.130:40971)
I20260808 10:06:47.962478 18016 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.130:56067
I20260808 10:06:47.965268 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 18304
I20260808 10:06:47.965396 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:47.965590 25226 external_mini_cluster.cc:1428] Setting key 626ead8baeb4309c49e8f7435daaab82
I20260808 10:06:47.966634 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--builtin_ntp_servers=127.24.162.148:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:48.049226 18501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:48.049423 18501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:48.049444 18501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:48.050601 18501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:48.050642 18501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:48.050668 18501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:48.052058 18501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:48.052302 18501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:48.052619 18501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:48.057678 18511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:48.057703 18506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:48.057747 18507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:48.057830 18501 server_base.cc:1034] running on GCE node
I20260808 10:06:48.058044 18501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:48.058291 18501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:48.059456 18501 hybrid_clock.cc:648] HybridClock initialized: now 1786183608059434 us; error 35 us; skew 500 ppm
I20260808 10:06:48.060706 18501 webserver.cc:460] Webserver started at http://127.24.162.131:40063/ using document root <none> and password file <none>
I20260808 10:06:48.061028 18501 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:48.061076 18501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:48.061169 18501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:48.062325 18501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "07c27835db1b4032ac74ee99b6dad980"
format_stamp: "Formatted at 2026-08-08 10:06:48 on dist-test-slave-nfb5"
server_key: "7fdc1fe38ffd4e2ba9da2a01ff933ce7"
server_key_iv: "00ba6dd2815807b53baa80178408a689"
server_key_version: "encryptionkey@0"
I20260808 10:06:48.062719 18501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "07c27835db1b4032ac74ee99b6dad980"
format_stamp: "Formatted at 2026-08-08 10:06:48 on dist-test-slave-nfb5"
server_key: "7fdc1fe38ffd4e2ba9da2a01ff933ce7"
server_key_iv: "00ba6dd2815807b53baa80178408a689"
server_key_version: "encryptionkey@0"
I20260808 10:06:48.064043 18501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:06:48.064782 18519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:48.064975 18501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:48.065048 18501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "07c27835db1b4032ac74ee99b6dad980"
format_stamp: "Formatted at 2026-08-08 10:06:48 on dist-test-slave-nfb5"
server_key: "7fdc1fe38ffd4e2ba9da2a01ff933ce7"
server_key_iv: "00ba6dd2815807b53baa80178408a689"
server_key_version: "encryptionkey@0"
I20260808 10:06:48.065101 18501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:48.084118 18501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:48.084403 18501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:48.084508 18501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:48.084695 18501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:48.085160 18501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:48.085219 18501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:48.085260 18501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:48.085292 18501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:48.091837 18501 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:42847
I20260808 10:06:48.091882 18686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:42847 every 8 connection(s)
I20260808 10:06:48.092228 18501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:48.097308 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.093007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:49169 (local address 127.24.162.190:36707)
0808 10:06:48.093079 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.093082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.093340 (+ 258us) server_negotiation.cc:407] Connection header received
0808 10:06:48.093858 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.093859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.093871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.093928 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:48.094583 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.095130 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.096084 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.096255 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.096304 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.097081 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:48.097082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:48.097082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:48.097086 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:48.097099 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:48.097103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:48.097146 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.097228 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.097249 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:06:48.097538 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.092986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36707 (local address 127.24.162.131:49169)
0808 10:06:48.093213 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.093251 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.093768 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.093980 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.093983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.093991 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:48.094513 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.094518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.095223 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.095224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.096009 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.096011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.096331 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.096332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.096438 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.096951 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:48.096951 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:48.096980 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:48.097001 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:48.097159 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:48.097163 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:48.097165 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.097216 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.097291 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
I20260808 10:06:48.098088 18688 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36707
I20260808 10:06:48.098186 18688 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:48.098345 18688 heartbeater.cc:511] Master 127.24.162.190:36707 requested a full tablet report, sending...
I20260808 10:06:48.098697 18016 ts_manager.cc:194] Registered new tserver with Master: 07c27835db1b4032ac74ee99b6dad980 (127.24.162.131:42847)
I20260808 10:06:48.099148 18016 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.131:49169
I20260808 10:06:48.102310 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 18501
I20260808 10:06:48.102437 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:48.102622 25226 external_mini_cluster.cc:1428] Setting key 55f635c9a5d7640183f0002bd5b916cd
I20260808 10:06:48.103579 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:48.107973 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.105011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.24.162.190:36707)
0808 10:06:48.105088 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.105090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.105188 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:06:48.105264 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.105265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.105275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.105326 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:48.105499 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.105971 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.106735 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.106985 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.107068 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.107740 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:48.107741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:48.107742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:48.107747 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:48.107760 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:48.107764 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:48.107812 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.107869 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.107890 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:06:48.109503 18016 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:48.111205 18016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:48.117161 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:42847 (local address 127.0.0.1:48186)
0808 10:06:48.113378 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.113417 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.113475 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.113952 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.113955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.113957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:48.114020 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.114022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.114952 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.114953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.116018 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.116020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.116878 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.116879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.116951 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.116964 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.117030 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.117084 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":9,"thread_start_us":78,"threads_started":1}
I20260808 10:06:48.117168 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.24.162.131:42847)
0808 10:06:48.113782 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.113786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.113796 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:48.113840 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.113841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.113853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.113943 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:48.114105 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.114836 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.116116 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.116790 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.116878 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.116964 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.117054 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.117091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260808 10:06:48.117654 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:42733 (local address 127.0.0.1:44752)
0808 10:06:48.113607 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.113616 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.113659 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.114105 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.114106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.114107 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:48.114151 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.114152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.115485 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.115486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.116544 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.116547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.117404 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.117406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.117477 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.117482 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.117531 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.117571 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260808 10:06:48.117674 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.24.162.129:42733)
0808 10:06:48.113810 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.113815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.113827 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:06:48.113870 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.113872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.113887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.114226 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:48.114287 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.115387 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.116628 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.117311 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.117379 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.117468 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.117555 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.117600 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":150,"threads_started":1}
I20260808 10:06:48.117578 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40971 (local address 127.0.0.1:48090)
0808 10:06:48.113597 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.113609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.113666 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.114060 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.114061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.114064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:48.114114 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.114116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.115509 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.115510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.116403 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.116406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.117352 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.117354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.117412 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.117416 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.117447 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.117489 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":81,"threads_started":1}
I20260808 10:06:48.117880 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.113408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.24.162.130:40971)
0808 10:06:48.113886 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.113891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.113899 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:06:48.113935 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.113937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.113950 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.114194 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:48.114260 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.115406 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.116492 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.117240 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.117624 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.117733 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.117760 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.117802 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":89,"threads_started":1}
I20260808 10:06:48.118259 18594 tablet_service.cc:1467] Processing CreateTablet for tablet c60cb02a9a97422ab8cb50d29e396555 (DEFAULT_TABLE table=test-table [id=9415179e11bd4c9fbf394f32e8e8a91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:48.118306 18196 tablet_service.cc:1467] Processing CreateTablet for tablet c60cb02a9a97422ab8cb50d29e396555 (DEFAULT_TABLE table=test-table [id=9415179e11bd4c9fbf394f32e8e8a91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:48.118556 18594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60cb02a9a97422ab8cb50d29e396555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:48.118520 18391 tablet_service.cc:1467] Processing CreateTablet for tablet c60cb02a9a97422ab8cb50d29e396555 (DEFAULT_TABLE table=test-table [id=9415179e11bd4c9fbf394f32e8e8a91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:48.118652 18196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60cb02a9a97422ab8cb50d29e396555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:48.118824 18391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60cb02a9a97422ab8cb50d29e396555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:48.121059 18707 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Bootstrap starting.
I20260808 10:06:48.121757 18707 tablet_bootstrap.cc:654] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:48.122313 18707 log.cc:824] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:48.122332 18709 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Bootstrap starting.
I20260808 10:06:48.122853 18709 tablet_bootstrap.cc:654] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:48.123123 18707 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: No bootstrap required, opened a new log
I20260808 10:06:48.123172 18709 log.cc:824] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:48.123204 18707 ts_tablet_manager.cc:1397] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:48.123519 18708 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Bootstrap starting.
I20260808 10:06:48.124271 18708 tablet_bootstrap.cc:654] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:48.124605 18708 log.cc:824] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:48.124989 18709 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: No bootstrap required, opened a new log
I20260808 10:06:48.125063 18709 ts_tablet_manager.cc:1397] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 10:06:48.125492 18708 tablet_bootstrap.cc:492] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: No bootstrap required, opened a new log
I20260808 10:06:48.125576 18708 ts_tablet_manager.cc:1397] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:48.126013 18707 raft_consensus.cc:348] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.126186 18707 raft_consensus.cc:374] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:48.126221 18707 raft_consensus.cc:729] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c27835db1b4032ac74ee99b6dad980, State: Initialized, Role: FOLLOWER
I20260808 10:06:48.126362 18707 consensus_queue.cc:260] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.126631 18707 ts_tablet_manager.cc:1428] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 10:06:48.126678 18688 heartbeater.cc:503] Master 127.24.162.190:36707 was elected leader, sending a full tablet report...
I20260808 10:06:48.127663 18709 raft_consensus.cc:348] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.127847 18709 raft_consensus.cc:374] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:48.127895 18709 raft_consensus.cc:729] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10417c8c58c04baf99ebafc075e648bf, State: Initialized, Role: FOLLOWER
I20260808 10:06:48.128011 18709 consensus_queue.cc:260] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.128062 18708 raft_consensus.cc:348] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.128203 18708 raft_consensus.cc:374] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:48.128232 18708 raft_consensus.cc:729] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58e8fa6b2d24f868ee5c36c16eaf52a, State: Initialized, Role: FOLLOWER
I20260808 10:06:48.128295 18709 ts_tablet_manager.cc:1428] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:06:48.128334 18708 consensus_queue.cc:260] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.128441 18497 heartbeater.cc:503] Master 127.24.162.190:36707 was elected leader, sending a full tablet report...
I20260808 10:06:48.128589 18708 ts_tablet_manager.cc:1428] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:48.128743 18298 heartbeater.cc:503] Master 127.24.162.190:36707 was elected leader, sending a full tablet report...
W20260808 10:06:48.206192 18499 tablet.cc:2369] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:48.208890 18716 raft_consensus.cc:482] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:48.209012 18716 raft_consensus.cc:504] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.209414 18716 leader_election.cc:288] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07c27835db1b4032ac74ee99b6dad980 (127.24.162.131:42847), 10417c8c58c04baf99ebafc075e648bf (127.24.162.130:40971)
I20260808 10:06:48.212357 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.209707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:49735 (local address 127.24.162.131:42847)
0808 10:06:48.209753 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.209755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.209764 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:06:48.209833 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.209833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.209842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.209914 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:48.210077 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.210628 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.211636 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.212006 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.212092 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.212100 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.212245 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.212285 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:06:48.212363 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.209615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:42847 (local address 127.24.162.129:49735)
0808 10:06:48.209680 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.209695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.209749 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.209919 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.209920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.209921 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:48.210004 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.210006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.210700 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.210701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.211534 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.211537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.212092 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.212093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.212170 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.212173 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.212241 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.212286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:06:48.212678 18623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60cb02a9a97422ab8cb50d29e396555" candidate_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c27835db1b4032ac74ee99b6dad980" is_pre_election: true
I20260808 10:06:48.212834 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.209720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:34967 (local address 127.24.162.130:40971)
0808 10:06:48.209770 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.209772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.209988 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:06:48.210070 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.210071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.210086 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.210171 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:48.210337 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.211107 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.212043 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.212447 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.212525 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.212534 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.212716 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.212765 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:06:48.212831 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.209624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:40971 (local address 127.24.162.129:34967)
0808 10:06:48.209911 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.209920 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:48.210000 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:48.210188 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:48.210189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:48.210191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:48.210265 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.210266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.211210 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.211211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.211946 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:48.211948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.212583 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.212584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:48.212656 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.212658 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:48.212704 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:06:48.212732 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":78,"thread_start_us":151,"threads_started":1}
I20260808 10:06:48.212845 18623 raft_consensus.cc:2454] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58e8fa6b2d24f868ee5c36c16eaf52a in term 0.
I20260808 10:06:48.213078 18125 leader_election.cc:302] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c27835db1b4032ac74ee99b6dad980, a58e8fa6b2d24f868ee5c36c16eaf52a; no voters:
I20260808 10:06:48.213235 18716 raft_consensus.cc:2781] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:48.213249 18429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60cb02a9a97422ab8cb50d29e396555" candidate_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10417c8c58c04baf99ebafc075e648bf" is_pre_election: true
I20260808 10:06:48.213339 18716 raft_consensus.cc:482] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:48.213379 18716 raft_consensus.cc:3037] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:48.213406 18429 raft_consensus.cc:2454] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58e8fa6b2d24f868ee5c36c16eaf52a in term 0.
I20260808 10:06:48.214156 18716 raft_consensus.cc:504] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.214316 18716 leader_election.cc:288] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [CANDIDATE]: Term 1 election: Requested vote from peers 07c27835db1b4032ac74ee99b6dad980 (127.24.162.131:42847), 10417c8c58c04baf99ebafc075e648bf (127.24.162.130:40971)
I20260808 10:06:48.214468 18623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60cb02a9a97422ab8cb50d29e396555" candidate_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c27835db1b4032ac74ee99b6dad980"
I20260808 10:06:48.214481 18429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60cb02a9a97422ab8cb50d29e396555" candidate_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10417c8c58c04baf99ebafc075e648bf"
I20260808 10:06:48.214555 18623 raft_consensus.cc:3037] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:48.214555 18429 raft_consensus.cc:3037] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:48.215492 18623 raft_consensus.cc:2454] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58e8fa6b2d24f868ee5c36c16eaf52a in term 1.
I20260808 10:06:48.215492 18429 raft_consensus.cc:2454] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58e8fa6b2d24f868ee5c36c16eaf52a in term 1.
I20260808 10:06:48.215721 18126 leader_election.cc:302] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10417c8c58c04baf99ebafc075e648bf, a58e8fa6b2d24f868ee5c36c16eaf52a; no voters:
I20260808 10:06:48.215834 18716 raft_consensus.cc:2781] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:48.216050 18716 raft_consensus.cc:686] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 1 LEADER]: Becoming Leader. State: Replica: a58e8fa6b2d24f868ee5c36c16eaf52a, State: Running, Role: LEADER
I20260808 10:06:48.216163 18716 consensus_queue.cc:237] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } }
I20260808 10:06:48.217125 18016 catalog_manager.cc:5495] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a reported cstate change: term changed from 0 to 1, leader changed from <none> to a58e8fa6b2d24f868ee5c36c16eaf52a (127.24.162.129). New cstate: current_term: 1 leader_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a58e8fa6b2d24f868ee5c36c16eaf52a" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 } health_report { overall_health: UNKNOWN } } }
I20260808 10:06:48.236518 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.234067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.24.162.129:42733)
0808 10:06:48.234117 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:48.234120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:48.234145 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:06:48.234188 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:48.234189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:48.234207 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:48.234285 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:48.234427 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.234887 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.235720 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:48.235915 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:48.235994 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:48.236167 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:48.236269 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:48.236366 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:48.236386 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 10:06:48.236407 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:06:48.238325 18623 raft_consensus.cc:1261] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 1 FOLLOWER]: Refusing update from remote peer a58e8fa6b2d24f868ee5c36c16eaf52a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:48.238343 18429 raft_consensus.cc:1261] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 1 FOLLOWER]: Refusing update from remote peer a58e8fa6b2d24f868ee5c36c16eaf52a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:48.238725 18718 consensus_queue.cc:1035] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c27835db1b4032ac74ee99b6dad980" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 42847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:48.238873 18716 consensus_queue.cc:1035] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10417c8c58c04baf99ebafc075e648bf" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 40971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:48.241202 18721 mvcc.cc:205] Tried to move back new op lower bound from 7316208059341185024 to 7316208059254132736. Current Snapshot: MvccSnapshot[applied={T|T < 7316208059341185024}]
I20260808 10:06:48.243462 18016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:48.243567 18016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:48.244367 18016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20409bee0f904a829daf18c058d2fb28: Sending DeleteTablet for 3 replicas of tablet c60cb02a9a97422ab8cb50d29e396555
I20260808 10:06:48.244733 18391 tablet_service.cc:1514] Processing DeleteTablet for tablet c60cb02a9a97422ab8cb50d29e396555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:48 UTC) from {username='slave'} at 127.0.0.1:48090
I20260808 10:06:48.244732 18196 tablet_service.cc:1514] Processing DeleteTablet for tablet c60cb02a9a97422ab8cb50d29e396555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:48 UTC) from {username='slave'} at 127.0.0.1:44752
I20260808 10:06:48.244813 18594 tablet_service.cc:1514] Processing DeleteTablet for tablet c60cb02a9a97422ab8cb50d29e396555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:48 UTC) from {username='slave'} at 127.0.0.1:48186
I20260808 10:06:48.245015 18743 tablet_replica.cc:330] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: stopping tablet replica
I20260808 10:06:48.245075 18740 tablet_replica.cc:330] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: stopping tablet replica
I20260808 10:06:48.245113 18743 raft_consensus.cc:2229] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:48.245170 18740 raft_consensus.cc:2229] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:48.245208 18743 raft_consensus.cc:2258] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:48.245261 18740 raft_consensus.cc:2258] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:48.245518 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 18105
I20260808 10:06:48.245713 18740 ts_tablet_manager.cc:1905] T c60cb02a9a97422ab8cb50d29e396555 P a58e8fa6b2d24f868ee5c36c16eaf52a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:48.246168 18739 tablet_replica.cc:330] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: stopping tablet replica
I20260808 10:06:48.246263 18739 raft_consensus.cc:2229] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:48.246354 18739 raft_consensus.cc:2258] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:48.248425 18739 ts_tablet_manager.cc:1905] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:48.248885 18743 ts_tablet_manager.cc:1905] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:48.250277 18739 ts_tablet_manager.cc:1918] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:48.250347 18739 log.cc:1196] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/c60cb02a9a97422ab8cb50d29e396555
I20260808 10:06:48.250590 18743 ts_tablet_manager.cc:1918] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:48.250622 18739 ts_tablet_manager.cc:1939] T c60cb02a9a97422ab8cb50d29e396555 P 10417c8c58c04baf99ebafc075e648bf: Deleting consensus metadata
I20260808 10:06:48.250655 18743 log.cc:1196] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/c60cb02a9a97422ab8cb50d29e396555
I20260808 10:06:48.250895 18743 ts_tablet_manager.cc:1939] T c60cb02a9a97422ab8cb50d29e396555 P 07c27835db1b4032ac74ee99b6dad980: Deleting consensus metadata
I20260808 10:06:48.251086 17996 catalog_manager.cc:4841] TS 10417c8c58c04baf99ebafc075e648bf (127.24.162.130:40971): tablet c60cb02a9a97422ab8cb50d29e396555 (table test-table [id=9415179e11bd4c9fbf394f32e8e8a91f]) successfully deleted
I20260808 10:06:48.251304 17995 catalog_manager.cc:4841] TS 07c27835db1b4032ac74ee99b6dad980 (127.24.162.131:42847): tablet c60cb02a9a97422ab8cb50d29e396555 (table test-table [id=9415179e11bd4c9fbf394f32e8e8a91f]) successfully deleted
W20260808 10:06:48.256937 17995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:42733 (error 108)
I20260808 10:06:48.257159 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 18304
I20260808 10:06:48.259409 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:48.257287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:42733 (local address 127.0.0.1:44760)
0808 10:06:48.259302 (+ 2015us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:48.259362 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:42733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1987}
W20260808 10:06:48.263732 17995 catalog_manager.cc:4584] TS a58e8fa6b2d24f868ee5c36c16eaf52a (127.24.162.129:42733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c60cb02a9a97422ab8cb50d29e396555: Network error: Client connection negotiation failed: client connection to 127.24.162.129:42733: connect: Connection refused (error 111)
I20260808 10:06:48.268616 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 18501
I20260808 10:06:48.276415 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 17973
2026-08-08T10:06:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (781 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:48 dist-test-slave-nfb5 krb5kdc[18749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:48 dist-test-slave-nfb5 krb5kdc[18749](info): set up 2 sockets
Aug 08 10:06:48 dist-test-slave-nfb5 krb5kdc[18749](info): commencing operation
krb5kdc: starting...
W20260808 10:06:49.493690 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:49 dist-test-slave-nfb5 krb5kdc[18749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:06:49.618371 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36369
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:49.701155 18768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:49.701372 18768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:49.701395 18768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:49.702487 18768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:49.702523 18768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:49.702538 18768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:49.702550 18768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:49.702562 18768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:49.704054 18768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:49.704268 18768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:49.704459 18768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:49.709358 18777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:49.709462 18780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:49.709367 18775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:49.709658 18768 server_base.cc:1034] running on GCE node
I20260808 10:06:49.709887 18768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:49.710125 18768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:49.711264 18768 hybrid_clock.cc:648] HybridClock initialized: now 1786183609711239 us; error 40 us; skew 500 ppm
Aug 08 10:06:49 dist-test-slave-nfb5 krb5kdc[18749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183609, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:49.713873 18768 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:06:49.714349 18768 webserver.cc:460] Webserver started at http://127.24.162.190:46671/ using document root <none> and password file <none>
I20260808 10:06:49.714658 18768 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:49.714707 18768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:49.714862 18768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:49.715922 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "e31b911e24b2498d91f83f916468e03c"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "ce05f9b72ca87846952743de06cdaa6b"
server_key_iv: "96529eff7423a04ce2ef4d91c3547def"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.716297 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "e31b911e24b2498d91f83f916468e03c"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "ce05f9b72ca87846952743de06cdaa6b"
server_key_iv: "96529eff7423a04ce2ef4d91c3547def"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.717609 18768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:06:49.718362 18789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:49.718538 18768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:49.718647 18768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "e31b911e24b2498d91f83f916468e03c"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "ce05f9b72ca87846952743de06cdaa6b"
server_key_iv: "96529eff7423a04ce2ef4d91c3547def"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.718721 18768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:49.734750 18768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:49.736797 18768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:49.736975 18768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:49.741403 18768 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36369
I20260808 10:06:49.741442 18863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36369 every 8 connection(s)
I20260808 10:06:49.741806 18768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:49.742686 18864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:49.743736 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 18768
I20260808 10:06:49.743858 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:49.744065 25226 external_mini_cluster.cc:1428] Setting key e42fd39d0682526cbf0d69f42ce78041
I20260808 10:06:49.745426 18864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: Bootstrap starting.
I20260808 10:06:49.746210 18864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:49.746536 18864 log.cc:824] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:49.747294 18864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: No bootstrap required, opened a new log
I20260808 10:06:49.749665 18864 raft_consensus.cc:348] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } }
I20260808 10:06:49.749871 18864 raft_consensus.cc:374] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:49.749915 18864 raft_consensus.cc:729] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31b911e24b2498d91f83f916468e03c, State: Initialized, Role: FOLLOWER
I20260808 10:06:49.750051 18864 consensus_queue.cc:260] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } }
I20260808 10:06:49.750142 18864 raft_consensus.cc:388] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:49.750178 18864 raft_consensus.cc:482] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:49.750224 18864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:49.750869 18864 raft_consensus.cc:504] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } }
I20260808 10:06:49.751034 18864 leader_election.cc:302] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e31b911e24b2498d91f83f916468e03c; no voters:
Aug 08 10:06:49 dist-test-slave-nfb5 krb5kdc[18749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:49.751245 18864 leader_election.cc:288] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:49.751438 18874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:49.751523 18864 sys_catalog.cc:564] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:49.751641 18874 raft_consensus.cc:686] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [term 1 LEADER]: Becoming Leader. State: Replica: e31b911e24b2498d91f83f916468e03c, State: Running, Role: LEADER
I20260808 10:06:49.751844 18874 consensus_queue.cc:237] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } }
W20260808 10:06:49.753551 18893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:06:49.753610 18893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:06:49.752749 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e31b911e24b2498d91f83f916468e03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } } }
I20260808 10:06:49.753759 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:49.753966 18871 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e31b911e24b2498d91f83f916468e03c. Latest consensus state: current_term: 1 leader_uuid: "e31b911e24b2498d91f83f916468e03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31b911e24b2498d91f83f916468e03c" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36369 } } }
I20260808 10:06:49.754146 18871 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:49.754201 18896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:49.754498 18896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:49.755407 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:49.744621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35978 (local address 127.24.162.190:36369)
0808 10:06:49.744835 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:49.744843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:49.744867 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:06:49.745272 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:49.745278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:49.745322 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:49.745434 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:49.746233 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.747274 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.748307 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.748714 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.748801 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:49.751753 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:49.751757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:49.751758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:49.751770 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:49.753791 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:49.754226 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:49.754230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:49.754230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:49.754336 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:49.754559 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:49.754560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:49.754561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:49.754805 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:49.755007 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:49.755082 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:06:49.755235 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":133,"threads_started":1}
I20260808 10:06:49.756132 18896 catalog_manager.cc:1347] Generated new cluster ID: d0b73950f127445a80307f8654b90288
I20260808 10:06:49.756189 18896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:49.778141 18896 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:49.778779 18896 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:49.793380 18896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: Generated new TSK 0
I20260808 10:06:49.793591 18896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:06:49.825637 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:49.914841 18902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:49.915174 18902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:49.915203 18902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:49.916323 18902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:49.916359 18902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:49.916385 18902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:49.917768 18902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:49.917958 18902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:49.918147 18902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:49.923170 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:49.923208 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:49.923189 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:49.923439 18902 server_base.cc:1034] running on GCE node
I20260808 10:06:49.923712 18902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:49.924027 18902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:49.925230 18902 hybrid_clock.cc:648] HybridClock initialized: now 1786183609925209 us; error 46 us; skew 500 ppm
Aug 08 10:06:49 dist-test-slave-nfb5 krb5kdc[18749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183609, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:49.927961 18902 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:06:49.928416 18902 webserver.cc:460] Webserver started at http://127.24.162.129:40321/ using document root <none> and password file <none>
I20260808 10:06:49.928613 18902 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:49.928653 18902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:49.928750 18902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:49.929733 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "2f2f2c4692034d32ae2be09a5342222d"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "996e2d65846f7588a22feb16031be6a1"
server_key_iv: "ad227ea20c0b432c0b08cd88beb90f8c"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.930038 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "2f2f2c4692034d32ae2be09a5342222d"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "996e2d65846f7588a22feb16031be6a1"
server_key_iv: "ad227ea20c0b432c0b08cd88beb90f8c"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.931439 18902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:49.932312 18918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:49.932487 18902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:49.932590 18902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "2f2f2c4692034d32ae2be09a5342222d"
format_stamp: "Formatted at 2026-08-08 10:06:49 on dist-test-slave-nfb5"
server_key: "996e2d65846f7588a22feb16031be6a1"
server_key_iv: "ad227ea20c0b432c0b08cd88beb90f8c"
server_key_version: "encryptionkey@0"
I20260808 10:06:49.932663 18902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:49.956333 18902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:49.958834 18902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:49.958971 18902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:49.959213 18902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:49.959540 18902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:49.959575 18902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:49.959597 18902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:49.959645 18902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:49.965665 18902 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:43247
I20260808 10:06:49.965718 19082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:43247 every 8 connection(s)
I20260808 10:06:49.966034 18902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:49.975673 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 18902
I20260808 10:06:49.975795 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:49.976012 25226 external_mini_cluster.cc:1428] Setting key b344074fae455fa28805c13c2931cc8b
Aug 08 10:06:49 dist-test-slave-nfb5 krb5kdc[18749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183609, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:49.979849 19086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:49.966717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36369 (local address 127.24.162.129:57479)
0808 10:06:49.966886 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:49.966957 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:49.967447 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:49.967649 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:49.967652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:49.968225 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:49.968415 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:49.968420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.969092 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.969093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:49.969878 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:49.969880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.970216 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.970217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:49.970310 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:49.975055 (+ 4745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:49.975064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:49.976723 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:49.978400 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:49.978409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:49.978411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:49.978647 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:49.978873 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:49.978874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:49.978874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:49.979033 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:49.979320 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:49.979325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:49.979467 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 10:06:49.979551 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:06:49.979671 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260808 10:06:49.979867 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:49.966766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:57479 (local address 127.24.162.190:36369)
0808 10:06:49.966847 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:49.966850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:49.967055 (+ 205us) server_negotiation.cc:407] Connection header received
0808 10:06:49.967529 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:49.967530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:49.967539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:49.967589 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:49.968492 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.969009 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.969950 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:49.970143 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:49.970195 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:49.976821 (+ 6626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:49.976823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:49.976823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:49.976831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:49.978300 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:49.978725 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:49.978726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:49.978727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:49.978796 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:49.979100 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:49.979101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:49.979101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:49.979249 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:49.979323 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:49.979620 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:06:49.979731 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:06:49.980389 19083 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36369
I20260808 10:06:49.980506 19083 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:49.980693 19083 heartbeater.cc:511] Master 127.24.162.190:36369 requested a full tablet report, sending...
I20260808 10:06:49.981244 18811 ts_manager.cc:194] Registered new tserver with Master: 2f2f2c4692034d32ae2be09a5342222d (127.24.162.129:43247)
I20260808 10:06:49.981897 18811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:57479
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:06:50.003579 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:50.086196 19091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:50.086401 19091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:50.086424 19091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:50.087587 19091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:50.087633 19091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:50.087663 19091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:50.089264 19091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:50.089521 19091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:50.089762 19091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:50.094856 19102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.094906 19098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.094959 19091 server_base.cc:1034] running on GCE node
W20260808 10:06:50.094869 19099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.095274 19091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:50.095479 19091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:50.096616 19091 hybrid_clock.cc:648] HybridClock initialized: now 1786183610096596 us; error 37 us; skew 500 ppm
Aug 08 10:06:50 dist-test-slave-nfb5 krb5kdc[18749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183610, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:50.099220 19091 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:06:50.099722 19091 webserver.cc:460] Webserver started at http://127.24.162.130:40791/ using document root <none> and password file <none>
I20260808 10:06:50.099956 19091 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:50.100003 19091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:50.100153 19091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:50.101236 19091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "09872ea8e2ab4499a42ae2a96c9d696d"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "9d71629d15743abc16c1d2e1614aca64"
server_key_iv: "b4868a1ff180c934126b40aeb1acdfe3"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.101585 19091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "09872ea8e2ab4499a42ae2a96c9d696d"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "9d71629d15743abc16c1d2e1614aca64"
server_key_iv: "b4868a1ff180c934126b40aeb1acdfe3"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.103018 19091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:06:50.103749 19110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.103924 19091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:50.104025 19091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "09872ea8e2ab4499a42ae2a96c9d696d"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "9d71629d15743abc16c1d2e1614aca64"
server_key_iv: "b4868a1ff180c934126b40aeb1acdfe3"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.104106 19091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:50.120393 19091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:50.122816 19091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:50.123006 19091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:50.123245 19091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:50.123558 19091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:50.123615 19091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.123667 19091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:50.123713 19091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.129814 19091 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:36535
I20260808 10:06:50.129864 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:36535 every 8 connection(s)
I20260808 10:06:50.130172 19091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:06:50 dist-test-slave-nfb5 krb5kdc[18749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183610, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:50.139358 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.130935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:47293 (local address 127.24.162.190:36369)
0808 10:06:50.131002 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.131005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.131207 (+ 202us) server_negotiation.cc:407] Connection header received
0808 10:06:50.131661 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.131662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.131673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.131727 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.132637 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.133130 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.134109 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.134294 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.134350 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.136799 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.136800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.136801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:50.136808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.138066 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.138434 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.138435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.138435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.138485 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.138758 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.138759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.138759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.138866 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.138982 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.139218 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.139294 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:06:50.139423 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 19091
I20260808 10:06:50.139521 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:50.139629 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.130868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36369 (local address 127.24.162.130:47293)
0808 10:06:50.131065 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.131101 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.131578 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.131784 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.131787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.132350 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:50.132548 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.132552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.133212 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.133212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.134029 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.134031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.134372 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.134373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.134476 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.135102 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:50.135109 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:50.136710 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:50.138144 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:50.138151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:50.138152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:50.138367 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:50.138544 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:50.138544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:50.138545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:50.138639 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:50.138986 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:50.138991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:50.139102 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.139190 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.139380 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
I20260808 10:06:50.139724 25226 external_mini_cluster.cc:1428] Setting key b75b48b73f5e10963cebf8cb4b60e04e
I20260808 10:06:50.140233 19278 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36369
I20260808 10:06:50.140357 19278 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:50.140537 19278 heartbeater.cc:511] Master 127.24.162.190:36369 requested a full tablet report, sending...
I20260808 10:06:50.141010 18810 ts_manager.cc:194] Registered new tserver with Master: 09872ea8e2ab4499a42ae2a96c9d696d (127.24.162.130:36535)
I20260808 10:06:50.141525 18810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:47293
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:06:50.166334 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:50.248924 19283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:50.249189 19283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:50.249233 19283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:50.250398 19283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:50.250466 19283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:50.250514 19283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:50.252310 19283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:50.252593 19283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:50.252838 19283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:50.258147 19289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.258147 19292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.258191 19290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.258490 19283 server_base.cc:1034] running on GCE node
I20260808 10:06:50.258703 19283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:50.258988 19283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:50.260138 19283 hybrid_clock.cc:648] HybridClock initialized: now 1786183610260113 us; error 42 us; skew 500 ppm
Aug 08 10:06:50 dist-test-slave-nfb5 krb5kdc[18749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183610, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:50.262756 19283 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:06:50.263284 19283 webserver.cc:460] Webserver started at http://127.24.162.131:39993/ using document root <none> and password file <none>
I20260808 10:06:50.263525 19283 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:50.263572 19283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:50.263734 19283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:50.264773 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "57936a5e21f04aa083246feea73b5579"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "bb52c03b358ac8c638b58a743e00a3ec"
server_key_iv: "627c0e32d54a62972ec53011ba1fdcb0"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.265132 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "57936a5e21f04aa083246feea73b5579"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "bb52c03b358ac8c638b58a743e00a3ec"
server_key_iv: "627c0e32d54a62972ec53011ba1fdcb0"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.266464 19283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:06:50.267280 19310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.267458 19283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.267550 19283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "57936a5e21f04aa083246feea73b5579"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "bb52c03b358ac8c638b58a743e00a3ec"
server_key_iv: "627c0e32d54a62972ec53011ba1fdcb0"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.267604 19283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:50.297341 19283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:50.299417 19283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:50.299530 19283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:50.299724 19283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:50.300098 19283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:50.300136 19283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.300203 19283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:50.300228 19283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.306083 19283 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:45045
I20260808 10:06:50.306121 19467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:45045 every 8 connection(s)
I20260808 10:06:50.306499 19283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:50.311719 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 19283
I20260808 10:06:50.311810 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:50.312036 25226 external_mini_cluster.cc:1428] Setting key 9178ea111fa0e2ec129fa05e142a89c6
Aug 08 10:06:50 dist-test-slave-nfb5 krb5kdc[18749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183610, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:50.315603 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.307310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:60829 (local address 127.24.162.190:36369)
0808 10:06:50.307401 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.307404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.307594 (+ 190us) server_negotiation.cc:407] Connection header received
0808 10:06:50.308141 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.308142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.308152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.308215 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.309099 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.309593 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.310536 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.310716 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.310766 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.313275 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.313276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.313277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:50.313283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.314446 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.314805 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.314806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.314807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.314860 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.315116 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.315116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.315117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.315235 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.315288 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.315482 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.315539 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:06:50.315872 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.307270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36369 (local address 127.24.162.131:60829)
0808 10:06:50.307460 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.307504 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.308053 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.308270 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.308273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.308835 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:50.309020 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.309024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.309672 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.309672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.310461 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.310463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.310788 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.310789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.310889 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.311361 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:50.311369 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:50.313196 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:50.314515 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:50.314521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:50.314522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:50.314735 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:50.314957 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:50.314958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:50.314958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:50.315058 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:50.315282 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:50.315286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:50.315395 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.315472 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.315588 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 10:06:50.316439 19468 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36369
I20260808 10:06:50.316546 19468 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:50.316718 19468 heartbeater.cc:511] Master 127.24.162.190:36369 requested a full tablet report, sending...
I20260808 10:06:50.317138 18810 ts_manager.cc:194] Registered new tserver with Master: 57936a5e21f04aa083246feea73b5579 (127.24.162.131:45045)
I20260808 10:06:50.317476 18810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:60829
I20260808 10:06:50.324026 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:50.331487 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.325360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.24.162.190:36369)
0808 10:06:50.325436 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.325438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.325540 (+ 102us) server_negotiation.cc:407] Connection header received
0808 10:06:50.325616 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.325617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.325627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.325672 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.326239 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.326713 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.327524 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.327720 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.327812 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.329049 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.329050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.329051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:50.329059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.330221 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.330621 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.330622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.330623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.330683 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:50.330907 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:50.330908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:50.330909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:50.331075 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.331197 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.331314 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.331380 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:06:50.332949 18810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:50.334489 18810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:50.341308 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.337134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.24.162.130:36535)
0808 10:06:50.337397 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.337400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.337409 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:06:50.337446 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.337450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.337489 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.337556 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:50.337837 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.338591 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.340130 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.340866 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.340989 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341071 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.341178 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.341219 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260808 10:06:50.341311 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.336869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:45045 (local address 127.0.0.1:57498)
0808 10:06:50.337022 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.337062 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.337132 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.337684 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.337685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.337686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:50.337755 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.337756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.339107 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.339108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.340165 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.340173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.341063 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.341064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.341135 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341139 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.341172 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.341220 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260808 10:06:50.341317 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.336871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:36535 (local address 127.0.0.1:55464)
0808 10:06:50.337220 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.337231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.337289 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.337653 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.337655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.337657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:50.337749 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.337751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.338705 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.338706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.340045 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.340048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.340986 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.340987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.341078 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341094 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.341165 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.341224 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260808 10:06:50.341679 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.337052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57498 (local address 127.24.162.131:45045)
0808 10:06:50.337351 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.337355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.337365 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:50.337406 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.337408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.337426 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.337514 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:50.337844 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.338999 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.340253 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.340957 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.341445 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341522 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.341551 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.341601 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":84,"threads_started":1}
I20260808 10:06:50.342113 19487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.336875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:43247 (local address 127.0.0.1:38090)
0808 10:06:50.337313 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.337322 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.337370 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.337681 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.337682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.337684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:50.337754 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.337756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.340241 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.340241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.340899 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.340901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.341860 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.341861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.341928 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341932 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.342003 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.342043 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":25,"thread_start_us":53,"threads_started":1}
I20260808 10:06:50.342347 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.337122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.24.162.129:43247)
0808 10:06:50.337293 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.337307 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.337435 (+ 128us) server_negotiation.cc:407] Connection header received
0808 10:06:50.337497 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.337500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.337518 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.337593 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:50.337849 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.339006 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.341037 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.341761 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.341837 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.341889 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.342218 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.342263 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260808 10:06:50.342548 19370 tablet_service.cc:1467] Processing CreateTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 (DEFAULT_TABLE table=test-table [id=c61c2d86e7ef490caf140abd2232c893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:50.342658 19183 tablet_service.cc:1467] Processing CreateTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 (DEFAULT_TABLE table=test-table [id=c61c2d86e7ef490caf140abd2232c893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:50.342957 19183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020fb7ae8c614648b652fc1a6b6f7883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:50.342959 19370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020fb7ae8c614648b652fc1a6b6f7883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:50.343581 18988 tablet_service.cc:1467] Processing CreateTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 (DEFAULT_TABLE table=test-table [id=c61c2d86e7ef490caf140abd2232c893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:50.343891 18988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020fb7ae8c614648b652fc1a6b6f7883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:50.346403 19493 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: Bootstrap starting.
I20260808 10:06:50.346534 19494 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Bootstrap starting.
I20260808 10:06:50.347136 19493 tablet_bootstrap.cc:654] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:50.347182 19494 tablet_bootstrap.cc:654] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:50.347657 19493 log.cc:824] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:50.347657 19494 log.cc:824] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:50.348052 19495 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Bootstrap starting.
I20260808 10:06:50.348510 19494 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: No bootstrap required, opened a new log
I20260808 10:06:50.348585 19494 ts_tablet_manager.cc:1397] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:50.348726 19495 tablet_bootstrap.cc:654] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:50.349272 19495 log.cc:824] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:50.349844 19493 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: No bootstrap required, opened a new log
I20260808 10:06:50.349913 19493 ts_tablet_manager.cc:1397] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:06:50.350068 19495 tablet_bootstrap.cc:492] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: No bootstrap required, opened a new log
I20260808 10:06:50.350140 19495 ts_tablet_manager.cc:1397] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:50.350746 19494 raft_consensus.cc:348] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.350977 19494 raft_consensus.cc:374] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:50.351025 19494 raft_consensus.cc:729] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09872ea8e2ab4499a42ae2a96c9d696d, State: Initialized, Role: FOLLOWER
I20260808 10:06:50.351141 19494 consensus_queue.cc:260] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.351467 19278 heartbeater.cc:503] Master 127.24.162.190:36369 was elected leader, sending a full tablet report...
I20260808 10:06:50.351480 19494 ts_tablet_manager.cc:1428] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:50.352051 19495 raft_consensus.cc:348] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.352223 19495 raft_consensus.cc:374] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:50.352311 19495 raft_consensus.cc:729] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57936a5e21f04aa083246feea73b5579, State: Initialized, Role: FOLLOWER
I20260808 10:06:50.352245 19493 raft_consensus.cc:348] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.352458 19495 consensus_queue.cc:260] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.352438 19493 raft_consensus.cc:374] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:50.352660 19493 raft_consensus.cc:729] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2f2c4692034d32ae2be09a5342222d, State: Initialized, Role: FOLLOWER
I20260808 10:06:50.352792 19495 ts_tablet_manager.cc:1428] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:50.352833 19493 consensus_queue.cc:260] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.352967 19468 heartbeater.cc:503] Master 127.24.162.190:36369 was elected leader, sending a full tablet report...
I20260808 10:06:50.353138 19493 ts_tablet_manager.cc:1428] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:50.353189 19083 heartbeater.cc:503] Master 127.24.162.190:36369 was elected leader, sending a full tablet report...
W20260808 10:06:50.380779 19279 tablet.cc:2369] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:50.440002 19504 raft_consensus.cc:482] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:50.440142 19504 raft_consensus.cc:504] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.440559 19504 leader_election.cc:288] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09872ea8e2ab4499a42ae2a96c9d696d (127.24.162.130:36535), 2f2f2c4692034d32ae2be09a5342222d (127.24.162.129:43247)
I20260808 10:06:50.443531 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.440763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:43247 (local address 127.24.162.131:50907)
0808 10:06:50.440827 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.440839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.440887 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.441071 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.441072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.441074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:50.441161 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.441163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.441854 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.441855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.442614 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.442616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.443264 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.443265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.443346 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.443349 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.443401 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.443432 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:06:50.443542 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.440854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:50907 (local address 127.24.162.129:43247)
0808 10:06:50.440902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.440905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.440915 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:50.440948 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.440949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.440959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.441036 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:50.441247 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.441779 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.442701 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.443178 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.443239 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.443246 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.443427 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.443475 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:06:50.443604 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.440864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:59621 (local address 127.24.162.130:36535)
0808 10:06:50.440904 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.440906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.441078 (+ 172us) server_negotiation.cc:407] Connection header received
0808 10:06:50.441116 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.441118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.441130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.441192 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:50.441389 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.441865 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.442792 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.443236 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.443312 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.443321 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.443495 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.443530 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:06:50.443647 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.440764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:36535 (local address 127.24.162.131:59621)
0808 10:06:50.440990 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.441001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.441061 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.441221 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.441222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.441223 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:50.441302 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.441303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.441947 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.441948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.442703 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.442705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.443332 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.443333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.443407 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.443411 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.443472 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.443509 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":12,"thread_start_us":132,"threads_started":1}
I20260808 10:06:50.443928 19016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020fb7ae8c614648b652fc1a6b6f7883" candidate_uuid: "57936a5e21f04aa083246feea73b5579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2f2c4692034d32ae2be09a5342222d" is_pre_election: true
I20260808 10:06:50.444031 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020fb7ae8c614648b652fc1a6b6f7883" candidate_uuid: "57936a5e21f04aa083246feea73b5579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" is_pre_election: true
I20260808 10:06:50.444113 19016 raft_consensus.cc:2454] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57936a5e21f04aa083246feea73b5579 in term 0.
I20260808 10:06:50.444185 19211 raft_consensus.cc:2454] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57936a5e21f04aa083246feea73b5579 in term 0.
I20260808 10:06:50.444360 19312 leader_election.cc:302] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2f2c4692034d32ae2be09a5342222d, 57936a5e21f04aa083246feea73b5579; no voters:
I20260808 10:06:50.444484 19504 raft_consensus.cc:2781] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:50.444562 19504 raft_consensus.cc:482] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:50.444599 19504 raft_consensus.cc:3037] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:50.445327 19504 raft_consensus.cc:504] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.445479 19504 leader_election.cc:288] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [CANDIDATE]: Term 1 election: Requested vote from peers 09872ea8e2ab4499a42ae2a96c9d696d (127.24.162.130:36535), 2f2f2c4692034d32ae2be09a5342222d (127.24.162.129:43247)
I20260808 10:06:50.445658 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020fb7ae8c614648b652fc1a6b6f7883" candidate_uuid: "57936a5e21f04aa083246feea73b5579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09872ea8e2ab4499a42ae2a96c9d696d"
I20260808 10:06:50.445704 19016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020fb7ae8c614648b652fc1a6b6f7883" candidate_uuid: "57936a5e21f04aa083246feea73b5579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2f2c4692034d32ae2be09a5342222d"
I20260808 10:06:50.445752 19211 raft_consensus.cc:3037] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:50.445772 19016 raft_consensus.cc:3037] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:50.446689 19211 raft_consensus.cc:2454] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57936a5e21f04aa083246feea73b5579 in term 1.
I20260808 10:06:50.446718 19016 raft_consensus.cc:2454] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57936a5e21f04aa083246feea73b5579 in term 1.
I20260808 10:06:50.446899 19312 leader_election.cc:302] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2f2c4692034d32ae2be09a5342222d, 57936a5e21f04aa083246feea73b5579; no voters:
I20260808 10:06:50.447014 19504 raft_consensus.cc:2781] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:50.447149 19504 raft_consensus.cc:686] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 1 LEADER]: Becoming Leader. State: Replica: 57936a5e21f04aa083246feea73b5579, State: Running, Role: LEADER
I20260808 10:06:50.447214 19504 consensus_queue.cc:237] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } }
I20260808 10:06:50.448204 18810 catalog_manager.cc:5495] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57936a5e21f04aa083246feea73b5579 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "57936a5e21f04aa083246feea73b5579" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57936a5e21f04aa083246feea73b5579" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 } health_report { overall_health: UNKNOWN } } }
I20260808 10:06:50.458468 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.456136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57504 (local address 127.24.162.131:45045)
0808 10:06:50.456202 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.456204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.456213 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:06:50.456245 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.456246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.456268 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.456363 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:50.456499 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.456941 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.457714 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.457903 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.457979 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.458134 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:50.458226 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:50.458343 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.458376 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.458386 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:06:50.460228 19211 raft_consensus.cc:1261] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 1 FOLLOWER]: Refusing update from remote peer 57936a5e21f04aa083246feea73b5579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:50.460228 19016 raft_consensus.cc:1261] T 020fb7ae8c614648b652fc1a6b6f7883 P 2f2f2c4692034d32ae2be09a5342222d [term 1 FOLLOWER]: Refusing update from remote peer 57936a5e21f04aa083246feea73b5579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:50.460588 19504 consensus_queue.cc:1035] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2f2c4692034d32ae2be09a5342222d" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 43247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:50.460777 19508 consensus_queue.cc:1035] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09872ea8e2ab4499a42ae2a96c9d696d" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:50.462949 19511 mvcc.cc:205] Tried to move back new op lower bound from 7316208068442329088 to 7316208068392423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316208068442329088}]
I20260808 10:06:50.463181 19512 mvcc.cc:205] Tried to move back new op lower bound from 7316208068442329088 to 7316208068392423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316208068442329088}]
I20260808 10:06:50.463402 19513 mvcc.cc:205] Tried to move back new op lower bound from 7316208068442329088 to 7316208068392423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316208068442329088}]
I20260808 10:06:50.465796 18811 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:50.465898 18811 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:50.466835 18811 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e31b911e24b2498d91f83f916468e03c: Sending DeleteTablet for 3 replicas of tablet 020fb7ae8c614648b652fc1a6b6f7883
I20260808 10:06:50.467196 18988 tablet_service.cc:1514] Processing DeleteTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:50 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:38090
I20260808 10:06:50.467196 19370 tablet_service.cc:1514] Processing DeleteTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:50 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:57498
I20260808 10:06:50.467262 19183 tablet_service.cc:1514] Processing DeleteTablet for tablet 020fb7ae8c614648b652fc1a6b6f7883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:50 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:55464
I20260808 10:06:50.467461 19526 tablet_replica.cc:330] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: stopping tablet replica
I20260808 10:06:50.467555 19526 raft_consensus.cc:2229] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:50.467653 19526 raft_consensus.cc:2258] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:50.467787 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 18902
I20260808 10:06:50.472931 19526 ts_tablet_manager.cc:1905] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:50.474495 19526 ts_tablet_manager.cc:1918] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:50.474587 19526 log.cc:1196] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/020fb7ae8c614648b652fc1a6b6f7883
I20260808 10:06:50.474852 19526 ts_tablet_manager.cc:1939] T 020fb7ae8c614648b652fc1a6b6f7883 P 09872ea8e2ab4499a42ae2a96c9d696d: Deleting consensus metadata
I20260808 10:06:50.475310 18790 catalog_manager.cc:4841] TS 09872ea8e2ab4499a42ae2a96c9d696d (127.24.162.130:36535): tablet 020fb7ae8c614648b652fc1a6b6f7883 (table test-table [id=c61c2d86e7ef490caf140abd2232c893]) successfully deleted
I20260808 10:06:50.480497 19525 tablet_replica.cc:330] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: stopping tablet replica
I20260808 10:06:50.480623 19525 raft_consensus.cc:2229] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:50.480790 19525 raft_consensus.cc:2258] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:50.481305 19525 ts_tablet_manager.cc:1905] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:50.483037 19525 ts_tablet_manager.cc:1918] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:50.483121 19525 log.cc:1196] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/020fb7ae8c614648b652fc1a6b6f7883
I20260808 10:06:50.483428 19525 ts_tablet_manager.cc:1939] T 020fb7ae8c614648b652fc1a6b6f7883 P 57936a5e21f04aa083246feea73b5579: Deleting consensus metadata
I20260808 10:06:50.483937 18793 catalog_manager.cc:4841] TS 57936a5e21f04aa083246feea73b5579 (127.24.162.131:45045): tablet 020fb7ae8c614648b652fc1a6b6f7883 (table test-table [id=c61c2d86e7ef490caf140abd2232c893]) successfully deleted
W20260808 10:06:50.484983 18791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:43247 (error 108)
I20260808 10:06:50.485229 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 19091
I20260808 10:06:50.487282 19487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.485392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:43247 (local address 127.0.0.1:38096)
0808 10:06:50.487175 (+ 1783us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.487229 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1757}
W20260808 10:06:50.487455 18791 catalog_manager.cc:4584] TS 2f2f2c4692034d32ae2be09a5342222d (127.24.162.129:43247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 020fb7ae8c614648b652fc1a6b6f7883: Network error: Client connection negotiation failed: client connection to 127.24.162.129:43247: connect: Connection refused (error 111)
I20260808 10:06:50.496479 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 19283
I20260808 10:06:50.507982 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 18768
2026-08-08T10:06:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2229 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T10:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:50Z Disabled control of system clock
I20260808 10:06:50.540198 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36871
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:37919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:50.621726 19539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:50.621932 19539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:50.621951 19539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:50.623030 19539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:50.623066 19539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:50.623080 19539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:50.623092 19539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:50.623104 19539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:50.624377 19539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:36871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:50.624569 19539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:50.624815 19539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:50.629609 19551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.629573 19548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.629583 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.630002 19539 server_base.cc:1034] running on GCE node
I20260808 10:06:50.630244 19539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:50.630474 19539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:50.631641 19539 hybrid_clock.cc:648] HybridClock initialized: now 1786183610631635 us; error 38 us; skew 500 ppm
I20260808 10:06:50.632876 19539 webserver.cc:460] Webserver started at http://127.24.162.190:45325/ using document root <none> and password file <none>
I20260808 10:06:50.633126 19539 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:50.633219 19539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:50.633363 19539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:50.634483 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "854116d7404e43dcae8c54f8b08d5460"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "576f5b2673210d4cb0be23fdf6891490"
server_key_iv: "0386e3ed4d34c6c99326e3ebd49cf23c"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.634867 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "854116d7404e43dcae8c54f8b08d5460"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "576f5b2673210d4cb0be23fdf6891490"
server_key_iv: "0386e3ed4d34c6c99326e3ebd49cf23c"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.636215 19539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:50.636972 19559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.637147 19539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:06:50.637266 19539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "854116d7404e43dcae8c54f8b08d5460"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "576f5b2673210d4cb0be23fdf6891490"
server_key_iv: "0386e3ed4d34c6c99326e3ebd49cf23c"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.637341 19539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:50.648818 19539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:50.649098 19539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:50.649250 19539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:50.653213 19539 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:36871
I20260808 10:06:50.653255 19636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:36871 every 8 connection(s)
I20260808 10:06:50.653600 19539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:50.654333 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 19539
I20260808 10:06:50.654457 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:50.654637 25226 external_mini_cluster.cc:1428] Setting key 7d45710c590b27669a9409d7dca33eba
I20260808 10:06:50.654670 19637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:50.658185 19637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: Bootstrap starting.
I20260808 10:06:50.658874 19637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:50.659219 19637 log.cc:824] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:50.660099 19637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: No bootstrap required, opened a new log
I20260808 10:06:50.660853 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.655239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.24.162.190:36871)
0808 10:06:50.655487 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.655496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.655528 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:06:50.655968 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.655977 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.656061 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.656206 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.656392 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.657728 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.658582 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.659235 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.659381 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.660138 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.660142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.660143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:50.660359 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.660399 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:50.660412 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.660477 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.660555 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.660631 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260808 10:06:50.662317 19637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } }
I20260808 10:06:50.662451 19637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:50.662472 19637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 854116d7404e43dcae8c54f8b08d5460, State: Initialized, Role: FOLLOWER
I20260808 10:06:50.662585 19637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } }
I20260808 10:06:50.662660 19637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:50.662683 19637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:50.662709 19637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:50.663327 19637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } }
I20260808 10:06:50.663420 19637 leader_election.cc:302] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 854116d7404e43dcae8c54f8b08d5460; no voters:
I20260808 10:06:50.663547 19637 leader_election.cc:288] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:50.663597 19644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:50.663805 19644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [term 1 LEADER]: Becoming Leader. State: Replica: 854116d7404e43dcae8c54f8b08d5460, State: Running, Role: LEADER
I20260808 10:06:50.663803 19637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:50.663986 19644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } }
I20260808 10:06:50.664351 19645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "854116d7404e43dcae8c54f8b08d5460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } } }
I20260808 10:06:50.664354 19647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 854116d7404e43dcae8c54f8b08d5460. Latest consensus state: current_term: 1 leader_uuid: "854116d7404e43dcae8c54f8b08d5460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "854116d7404e43dcae8c54f8b08d5460" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 36871 } } }
I20260808 10:06:50.664469 19647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:50.664651 19645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:50.664994 19654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:50.665510 19654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:50.667167 19654 catalog_manager.cc:1347] Generated new cluster ID: 102b9bd46e89445c91d61b4fbe1a9443
I20260808 10:06:50.667248 19654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:50.675175 19654 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:50.675663 19654 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:50.677980 19654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: Generated new TSK 0
I20260808 10:06:50.678205 19654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:50.679595 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--builtin_ntp_servers=127.24.162.148:37919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:50.777904 19675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:50.778106 19675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:50.778127 19675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:50.779335 19675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:50.779374 19675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:50.779400 19675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:50.780773 19675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:50.781016 19675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:50.781347 19675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:50.787230 19685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.787237 19684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.787277 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.787827 19675 server_base.cc:1034] running on GCE node
I20260808 10:06:50.788038 19675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:50.788326 19675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:50.789513 19675 hybrid_clock.cc:648] HybridClock initialized: now 1786183610789499 us; error 27 us; skew 500 ppm
I20260808 10:06:50.791055 19675 webserver.cc:460] Webserver started at http://127.24.162.129:44929/ using document root <none> and password file <none>
I20260808 10:06:50.791358 19675 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:50.791430 19675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:50.791579 19675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:50.792995 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "505250bb16f64981a4120b61e158e469"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "09c7da3fc6151345b1bd176b19e13956"
server_key_iv: "8cc2dd2dc05227749f9880daeba93667"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.793418 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "505250bb16f64981a4120b61e158e469"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "09c7da3fc6151345b1bd176b19e13956"
server_key_iv: "8cc2dd2dc05227749f9880daeba93667"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.795238 19675 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 10:06:50.796339 19694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.796564 19675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:50.796715 19675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "505250bb16f64981a4120b61e158e469"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "09c7da3fc6151345b1bd176b19e13956"
server_key_iv: "8cc2dd2dc05227749f9880daeba93667"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.796828 19675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:50.811775 19675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:50.812125 19675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:50.812276 19675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:50.812493 19675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:50.812822 19675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:50.812879 19675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.812922 19675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:50.812950 19675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.818693 19675 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:37193
I20260808 10:06:50.818770 19857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:37193 every 8 connection(s)
I20260808 10:06:50.819115 19675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:50.824383 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.819843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:46369 (local address 127.24.162.190:36871)
0808 10:06:50.819916 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.819918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.820167 (+ 249us) server_negotiation.cc:407] Connection header received
0808 10:06:50.820722 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.820723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.820734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.820799 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.821476 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.821981 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.823024 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.823221 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.823300 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.824062 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.824063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.824063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:50.824068 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.824083 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:50.824087 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.824155 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.824253 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.824294 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:06:50.824584 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.819836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36871 (local address 127.24.162.129:46369)
0808 10:06:50.820038 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.820071 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.820582 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.820880 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.820883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.820892 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:50.821378 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.821382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.822082 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.822083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.822866 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.822868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.823309 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.823310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.823447 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.823922 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:50.823922 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:50.823949 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:50.823971 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:50.824157 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:50.824164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:50.824168 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.824253 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.824355 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260808 10:06:50.825157 19858 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36871
I20260808 10:06:50.825234 19858 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:50.825429 19858 heartbeater.cc:511] Master 127.24.162.190:36871 requested a full tablet report, sending...
I20260808 10:06:50.825960 19583 ts_manager.cc:194] Registered new tserver with Master: 505250bb16f64981a4120b61e158e469 (127.24.162.129:37193)
I20260808 10:06:50.825997 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 19675
I20260808 10:06:50.826099 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:50.826311 25226 external_mini_cluster.cc:1428] Setting key 23edf015ec3f396f9b973d4133cb137c
I20260808 10:06:50.826845 19583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.129:46369
I20260808 10:06:50.827358 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--builtin_ntp_servers=127.24.162.148:37919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:50.909536 19862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:50.909772 19862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:50.909812 19862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:50.910946 19862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:50.911011 19862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:50.911055 19862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:50.912422 19862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:50.912679 19862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:50.912986 19862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:50.917909 19870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:50.917929 19868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.918032 19862 server_base.cc:1034] running on GCE node
W20260808 10:06:50.917922 19867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:50.918377 19862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:50.918659 19862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:50.919804 19862 hybrid_clock.cc:648] HybridClock initialized: now 1786183610919794 us; error 31 us; skew 500 ppm
I20260808 10:06:50.921270 19862 webserver.cc:460] Webserver started at http://127.24.162.130:36373/ using document root <none> and password file <none>
I20260808 10:06:50.921511 19862 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:50.921559 19862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:50.921730 19862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:50.922828 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "ce48ae530099403a90aedd4ed22d0798"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "2a8a960f96c229e74d25afd6332a2c41"
server_key_iv: "c43e3ecb7b9d891cb5a9b4c1224d80ad"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.923260 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "ce48ae530099403a90aedd4ed22d0798"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "2a8a960f96c229e74d25afd6332a2c41"
server_key_iv: "c43e3ecb7b9d891cb5a9b4c1224d80ad"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.924513 19862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:50.925203 19879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.925388 19862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:50.925478 19862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "ce48ae530099403a90aedd4ed22d0798"
format_stamp: "Formatted at 2026-08-08 10:06:50 on dist-test-slave-nfb5"
server_key: "2a8a960f96c229e74d25afd6332a2c41"
server_key_iv: "c43e3ecb7b9d891cb5a9b4c1224d80ad"
server_key_version: "encryptionkey@0"
I20260808 10:06:50.925555 19862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:50.935784 19862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:50.936076 19862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:50.936231 19862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:50.936445 19862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:50.936789 19862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:50.936844 19862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.936887 19862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:50.936915 19862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:50.943027 19862 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:41007
I20260808 10:06:50.943094 20043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:41007 every 8 connection(s)
I20260808 10:06:50.943413 19862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:50.948314 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.944109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:33239 (local address 127.24.162.190:36871)
0808 10:06:50.944200 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:50.944203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:50.944394 (+ 191us) server_negotiation.cc:407] Connection header received
0808 10:06:50.944864 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:50.944865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:50.944876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:50.944920 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:50.945572 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.946051 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.947031 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.947222 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.947277 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.948072 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:50.948073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:50.948073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:50.948078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:50.948092 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:50.948095 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:50.948139 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:50.948233 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 10:06:50.948254 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:06:50.948570 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:50.944076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36871 (local address 127.24.162.130:33239)
0808 10:06:50.944275 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:50.944307 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:50.944768 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:50.944990 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:50.944994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:50.945002 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:50.945501 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.945505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.946131 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.946132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.946891 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:50.946893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:50.947297 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:50.947298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:50.947407 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:50.947917 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:50.947918 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:50.947960 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:50.947984 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:50.948161 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:50.948165 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:50.948168 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:50.948222 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:06:50.948304 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260808 10:06:50.949174 20044 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36871
I20260808 10:06:50.949257 20044 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:50.949396 20044 heartbeater.cc:511] Master 127.24.162.190:36871 requested a full tablet report, sending...
I20260808 10:06:50.949746 19583 ts_manager.cc:194] Registered new tserver with Master: ce48ae530099403a90aedd4ed22d0798 (127.24.162.130:41007)
I20260808 10:06:50.950116 19583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.130:33239
I20260808 10:06:50.952452 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 19862
I20260808 10:06:50.952544 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:50.952798 25226 external_mini_cluster.cc:1428] Setting key 00a0bc25bce803cd670f85fc1900066b
I20260808 10:06:50.953735 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--builtin_ntp_servers=127.24.162.148:37919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:51.035611 20053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:51.035811 20053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:51.035835 20053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:51.036893 20053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:51.036928 20053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:51.036952 20053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:51.038218 20053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:37919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:51.038393 20053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:51.038702 20053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:51.043814 20062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:51.043929 20053 server_base.cc:1034] running on GCE node
W20260808 10:06:51.043800 20060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:51.043996 20066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:51.044224 20053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:51.044509 20053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:51.045691 20053 hybrid_clock.cc:648] HybridClock initialized: now 1786183611045665 us; error 42 us; skew 500 ppm
I20260808 10:06:51.047057 20053 webserver.cc:460] Webserver started at http://127.24.162.131:44819/ using document root <none> and password file <none>
I20260808 10:06:51.047267 20053 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:51.047312 20053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:51.047405 20053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:51.048382 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "3306c678f773488a99a147bee83edd21"
format_stamp: "Formatted at 2026-08-08 10:06:51 on dist-test-slave-nfb5"
server_key: "ba7e1e7e5bec29847a895d8dc299de99"
server_key_iv: "d13991b4a04322cf402e07c0c5b83141"
server_key_version: "encryptionkey@0"
I20260808 10:06:51.048743 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "3306c678f773488a99a147bee83edd21"
format_stamp: "Formatted at 2026-08-08 10:06:51 on dist-test-slave-nfb5"
server_key: "ba7e1e7e5bec29847a895d8dc299de99"
server_key_iv: "d13991b4a04322cf402e07c0c5b83141"
server_key_version: "encryptionkey@0"
I20260808 10:06:51.050083 20053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:51.050805 20073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:51.050984 20053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:51.051075 20053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "3306c678f773488a99a147bee83edd21"
format_stamp: "Formatted at 2026-08-08 10:06:51 on dist-test-slave-nfb5"
server_key: "ba7e1e7e5bec29847a895d8dc299de99"
server_key_iv: "d13991b4a04322cf402e07c0c5b83141"
server_key_version: "encryptionkey@0"
I20260808 10:06:51.051131 20053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:51.063664 20053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:51.063910 20053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:51.064100 20053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:51.064277 20053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:51.064600 20053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:51.064636 20053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:51.064658 20053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:51.064671 20053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:51.070240 20053 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:36501
I20260808 10:06:51.070314 20249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:36501 every 8 connection(s)
I20260808 10:06:51.070641 20053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:51.075627 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.071415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:57491 (local address 127.24.162.190:36871)
0808 10:06:51.071466 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.071468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.071675 (+ 207us) server_negotiation.cc:407] Connection header received
0808 10:06:51.072188 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.072189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.072201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.072260 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:51.072918 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.073409 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.074311 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.074491 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.074546 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.075347 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:51.075348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:51.075349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:51.075353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:51.075367 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:51.075370 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:51.075418 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.075544 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.075567 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:06:51.075731 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.071391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:36871 (local address 127.24.162.131:57491)
0808 10:06:51.071575 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.071607 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.072065 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.072307 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.072310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.072320 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:51.072799 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.072803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.073488 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.073489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.074235 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.074237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.074563 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.074564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.074667 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.075212 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:51.075212 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:51.075241 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:51.075263 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:51.075421 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:51.075423 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:51.075425 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.075471 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.075546 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260808 10:06:51.076179 20250 heartbeater.cc:347] Connected to a master server at 127.24.162.190:36871
I20260808 10:06:51.076285 20250 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:51.076483 20250 heartbeater.cc:511] Master 127.24.162.190:36871 requested a full tablet report, sending...
I20260808 10:06:51.076845 19583 ts_manager.cc:194] Registered new tserver with Master: 3306c678f773488a99a147bee83edd21 (127.24.162.131:36501)
I20260808 10:06:51.077216 19583 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.131:57491
I20260808 10:06:51.079115 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 20053
I20260808 10:06:51.079228 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:51.079447 25226 external_mini_cluster.cc:1428] Setting key 9054345471c603ae50a377a7e8b3f4b3
I20260808 10:06:51.080667 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:51.084873 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.082071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.24.162.190:36871)
0808 10:06:51.082133 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.082136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.082271 (+ 135us) server_negotiation.cc:407] Connection header received
0808 10:06:51.082308 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.082309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.082318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.082358 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:51.082484 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.082954 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.083641 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.083828 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.083883 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.084559 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:51.084560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:51.084561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:51.084564 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:51.084576 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:51.084580 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:51.084654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.084739 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.084770 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:06:51.086582 19583 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:51.088202 19583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:51.095093 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.24.162.129:37193)
0808 10:06:51.091083 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.091087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.091411 (+ 324us) server_negotiation.cc:407] Connection header received
0808 10:06:51.091459 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.091461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.091478 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.091564 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:51.091769 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092821 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.093929 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.094649 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094734 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.094829 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.094967 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.095006 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":140,"threads_started":1}
I20260808 10:06:51.095094 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37193 (local address 127.0.0.1:45164)
0808 10:06:51.091325 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.091336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.091389 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.091638 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.091639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.091642 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:51.091683 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.091685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092901 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.092902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.093842 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.093845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.094744 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.094803 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.094814 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.094928 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.094986 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260808 10:06:51.095171 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:36501 (local address 127.0.0.1:50324)
0808 10:06:51.090650 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.090688 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.090754 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.091242 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.091245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.091247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:51.091323 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.091325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092750 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.092751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.094023 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.094026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.094886 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.094962 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.094965 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.095006 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.095037 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":55,"thread_start_us":67,"threads_started":1}
I20260808 10:06:51.096032 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:41007 (local address 127.0.0.1:57418)
0808 10:06:51.091467 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.091476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.091512 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.091817 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.091818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.091819 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:51.091857 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.091860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092701 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.092701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.094023 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.094026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.095780 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.095781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.095845 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.095849 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.095905 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.095952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":43,"thread_start_us":155,"threads_started":1}
I20260808 10:06:51.096066 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.24.162.131:36501)
0808 10:06:51.091062 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.091067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.091077 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:51.091112 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.091116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.091134 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.091233 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:51.091446 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092619 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094115 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.094800 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094869 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.095928 (+ 1059us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.095952 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.095989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":117,"threads_started":1}
I20260808 10:06:51.096139 19760 tablet_service.cc:1467] Processing CreateTablet for tablet 18a86884e3a245d7a3eb065b2513f233 (DEFAULT_TABLE table=test-table [id=33f850b3f60c4fddb163c219de00de45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:51.096374 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.090853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57418 (local address 127.24.162.130:41007)
0808 10:06:51.091635 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.091639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.091648 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:06:51.091691 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.091693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.091709 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.091816 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:51.091915 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.092597 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.094977 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.095699 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.095779 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.095860 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.096245 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.096287 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":105,"threads_started":1}
I20260808 10:06:51.096534 19760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a86884e3a245d7a3eb065b2513f233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:51.096639 20150 tablet_service.cc:1467] Processing CreateTablet for tablet 18a86884e3a245d7a3eb065b2513f233 (DEFAULT_TABLE table=test-table [id=33f850b3f60c4fddb163c219de00de45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:51.096966 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a86884e3a245d7a3eb065b2513f233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:51.099448 19945 tablet_service.cc:1467] Processing CreateTablet for tablet 18a86884e3a245d7a3eb065b2513f233 (DEFAULT_TABLE table=test-table [id=33f850b3f60c4fddb163c219de00de45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:51.099812 19945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a86884e3a245d7a3eb065b2513f233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:51.100344 20273 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: Bootstrap starting.
I20260808 10:06:51.101112 20273 tablet_bootstrap.cc:654] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:51.101501 20273 log.cc:824] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:51.101876 20274 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Bootstrap starting.
I20260808 10:06:51.102273 20273 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: No bootstrap required, opened a new log
I20260808 10:06:51.102389 20273 ts_tablet_manager.cc:1397] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:51.102586 20274 tablet_bootstrap.cc:654] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:51.102945 20274 log.cc:824] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:51.103590 20279 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Bootstrap starting.
I20260808 10:06:51.103845 20274 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: No bootstrap required, opened a new log
I20260808 10:06:51.103927 20274 ts_tablet_manager.cc:1397] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:51.104233 20279 tablet_bootstrap.cc:654] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:51.104552 20279 log.cc:824] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:51.105032 20273 raft_consensus.cc:348] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.105212 20273 raft_consensus.cc:374] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:51.105258 20273 raft_consensus.cc:729] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505250bb16f64981a4120b61e158e469, State: Initialized, Role: FOLLOWER
I20260808 10:06:51.105387 20279 tablet_bootstrap.cc:492] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: No bootstrap required, opened a new log
I20260808 10:06:51.105372 20273 consensus_queue.cc:260] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.105460 20279 ts_tablet_manager.cc:1397] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:51.105638 20273 ts_tablet_manager.cc:1428] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 10:06:51.105773 19858 heartbeater.cc:503] Master 127.24.162.190:36871 was elected leader, sending a full tablet report...
I20260808 10:06:51.106551 20274 raft_consensus.cc:348] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.106714 20274 raft_consensus.cc:374] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:51.106752 20274 raft_consensus.cc:729] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3306c678f773488a99a147bee83edd21, State: Initialized, Role: FOLLOWER
I20260808 10:06:51.106855 20274 consensus_queue.cc:260] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.107115 20274 ts_tablet_manager.cc:1428] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 10:06:51.107263 20250 heartbeater.cc:503] Master 127.24.162.190:36871 was elected leader, sending a full tablet report...
I20260808 10:06:51.108026 20279 raft_consensus.cc:348] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.108139 20279 raft_consensus.cc:374] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:51.108161 20279 raft_consensus.cc:729] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce48ae530099403a90aedd4ed22d0798, State: Initialized, Role: FOLLOWER
I20260808 10:06:51.108292 20279 consensus_queue.cc:260] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.108548 20279 ts_tablet_manager.cc:1428] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:51.108572 20044 heartbeater.cc:503] Master 127.24.162.190:36871 was elected leader, sending a full tablet report...
I20260808 10:06:51.162900 20285 raft_consensus.cc:482] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:51.163091 20285 raft_consensus.cc:504] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.163443 20285 leader_election.cc:288] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3306c678f773488a99a147bee83edd21 (127.24.162.131:36501), 505250bb16f64981a4120b61e158e469 (127.24.162.129:37193)
I20260808 10:06:51.167084 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.163611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:36501 (local address 127.24.162.130:52667)
0808 10:06:51.163662 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.163675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.163768 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.163990 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.163991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.163993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:51.164068 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.164069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.164943 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.164944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.166130 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.166132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.166839 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.166839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.166892 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.166895 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.166952 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.166984 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:06:51.167104 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.163690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:52667 (local address 127.24.162.131:36501)
0808 10:06:51.163742 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.163744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.163754 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:51.163885 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.163887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.163898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.163985 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:51.164142 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.164867 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.166215 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.166750 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.166829 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.166837 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.166978 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.167028 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:06:51.167307 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.163618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37193 (local address 127.24.162.130:50917)
0808 10:06:51.163912 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.163928 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:51.163991 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:51.164143 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:51.164144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:51.164145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:51.164234 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.164236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.165106 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.165107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.166258 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:51.166260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.167062 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.167063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:51.167141 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.167145 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:51.167204 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:06:51.167237 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":166,"thread_start_us":160,"threads_started":1}
I20260808 10:06:51.167322 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.163759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:50917 (local address 127.24.162.129:37193)
0808 10:06:51.163851 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.163853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.163981 (+ 128us) server_negotiation.cc:407] Connection header received
0808 10:06:51.164045 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.164046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.164057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.164137 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:51.164316 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.165025 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.166339 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.166974 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.167042 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.167051 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.167212 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.167256 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:06:51.167495 20178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a86884e3a245d7a3eb065b2513f233" candidate_uuid: "ce48ae530099403a90aedd4ed22d0798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3306c678f773488a99a147bee83edd21" is_pre_election: true
I20260808 10:06:51.167670 20178 raft_consensus.cc:2454] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce48ae530099403a90aedd4ed22d0798 in term 0.
I20260808 10:06:51.167677 19790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a86884e3a245d7a3eb065b2513f233" candidate_uuid: "ce48ae530099403a90aedd4ed22d0798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505250bb16f64981a4120b61e158e469" is_pre_election: true
I20260808 10:06:51.167846 19790 raft_consensus.cc:2454] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce48ae530099403a90aedd4ed22d0798 in term 0.
I20260808 10:06:51.167892 19884 leader_election.cc:302] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3306c678f773488a99a147bee83edd21, ce48ae530099403a90aedd4ed22d0798; no voters:
I20260808 10:06:51.168037 20285 raft_consensus.cc:2781] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:51.168088 20285 raft_consensus.cc:482] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:51.168104 20285 raft_consensus.cc:3037] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:51.168808 20285 raft_consensus.cc:504] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.168970 20285 leader_election.cc:288] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [CANDIDATE]: Term 1 election: Requested vote from peers 3306c678f773488a99a147bee83edd21 (127.24.162.131:36501), 505250bb16f64981a4120b61e158e469 (127.24.162.129:37193)
I20260808 10:06:51.169148 20178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a86884e3a245d7a3eb065b2513f233" candidate_uuid: "ce48ae530099403a90aedd4ed22d0798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3306c678f773488a99a147bee83edd21"
I20260808 10:06:51.169242 20178 raft_consensus.cc:3037] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:51.169989 20178 raft_consensus.cc:2454] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce48ae530099403a90aedd4ed22d0798 in term 1.
I20260808 10:06:51.170122 19790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a86884e3a245d7a3eb065b2513f233" candidate_uuid: "ce48ae530099403a90aedd4ed22d0798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505250bb16f64981a4120b61e158e469"
I20260808 10:06:51.170188 19884 leader_election.cc:302] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3306c678f773488a99a147bee83edd21, ce48ae530099403a90aedd4ed22d0798; no voters:
I20260808 10:06:51.170266 20285 raft_consensus.cc:2781] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:51.170226 19790 raft_consensus.cc:3037] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:51.170424 20285 raft_consensus.cc:686] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 1 LEADER]: Becoming Leader. State: Replica: ce48ae530099403a90aedd4ed22d0798, State: Running, Role: LEADER
I20260808 10:06:51.170529 20285 consensus_queue.cc:237] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } }
I20260808 10:06:51.171234 19790 raft_consensus.cc:2454] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce48ae530099403a90aedd4ed22d0798 in term 1.
I20260808 10:06:51.171491 19583 catalog_manager.cc:5495] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce48ae530099403a90aedd4ed22d0798 (127.24.162.130). New cstate: current_term: 1 leader_uuid: "ce48ae530099403a90aedd4ed22d0798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce48ae530099403a90aedd4ed22d0798" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41007 } health_report { overall_health: HEALTHY } } }
I20260808 10:06:51.187469 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.185319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.24.162.130:41007)
0808 10:06:51.185379 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:51.185382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:51.185395 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:06:51.185423 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:51.185424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:51.185440 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:51.185507 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:51.185620 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.186064 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.186804 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:51.187037 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:51.187091 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:51.187218 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:51.187300 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:51.187371 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:51.187391 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 10:06:51.187398 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:06:51.189201 19790 raft_consensus.cc:1261] T 18a86884e3a245d7a3eb065b2513f233 P 505250bb16f64981a4120b61e158e469 [term 1 FOLLOWER]: Refusing update from remote peer ce48ae530099403a90aedd4ed22d0798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:51.189217 20178 raft_consensus.cc:1261] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 1 FOLLOWER]: Refusing update from remote peer ce48ae530099403a90aedd4ed22d0798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:51.189570 20285 consensus_queue.cc:1035] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "505250bb16f64981a4120b61e158e469" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:51.189738 20287 consensus_queue.cc:1035] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3306c678f773488a99a147bee83edd21" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:51.192165 20293 mvcc.cc:205] Tried to move back new op lower bound from 7316208071428390912 to 7316208071355195392. Current Snapshot: MvccSnapshot[applied={T|T < 7316208071428390912}]
I20260808 10:06:51.192373 20296 mvcc.cc:205] Tried to move back new op lower bound from 7316208071428390912 to 7316208071355195392. Current Snapshot: MvccSnapshot[applied={T|T < 7316208071428390912}]
I20260808 10:06:51.192997 20295 mvcc.cc:205] Tried to move back new op lower bound from 7316208071428390912 to 7316208071355195392. Current Snapshot: MvccSnapshot[applied={T|T < 7316208071428390912}]
I20260808 10:06:51.194759 19583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:51.194864 19583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:51.195709 19583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 854116d7404e43dcae8c54f8b08d5460: Sending DeleteTablet for 3 replicas of tablet 18a86884e3a245d7a3eb065b2513f233
I20260808 10:06:51.196086 19760 tablet_service.cc:1514] Processing DeleteTablet for tablet 18a86884e3a245d7a3eb065b2513f233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:51 UTC) from {username='slave'} at 127.0.0.1:45164
I20260808 10:06:51.196184 19945 tablet_service.cc:1514] Processing DeleteTablet for tablet 18a86884e3a245d7a3eb065b2513f233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:51 UTC) from {username='slave'} at 127.0.0.1:57418
I20260808 10:06:51.196311 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet 18a86884e3a245d7a3eb065b2513f233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:51 UTC) from {username='slave'} at 127.0.0.1:50324
I20260808 10:06:51.196461 20306 tablet_replica.cc:330] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: stopping tablet replica
I20260808 10:06:51.196532 20307 tablet_replica.cc:330] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: stopping tablet replica
I20260808 10:06:51.196635 20307 raft_consensus.cc:2229] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:51.196637 20306 raft_consensus.cc:2229] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:51.196748 20307 raft_consensus.cc:2258] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:51.196738 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 19675
I20260808 10:06:51.196868 20306 raft_consensus.cc:2258] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:51.197208 20307 ts_tablet_manager.cc:1905] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:51.199455 20307 ts_tablet_manager.cc:1918] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:51.199544 20307 log.cc:1196] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/18a86884e3a245d7a3eb065b2513f233
I20260808 10:06:51.199858 20307 ts_tablet_manager.cc:1939] T 18a86884e3a245d7a3eb065b2513f233 P 3306c678f773488a99a147bee83edd21: Deleting consensus metadata
I20260808 10:06:51.200322 19567 catalog_manager.cc:4841] TS 3306c678f773488a99a147bee83edd21 (127.24.162.131:36501): tablet 18a86884e3a245d7a3eb065b2513f233 (table test-table [id=33f850b3f60c4fddb163c219de00de45]) successfully deleted
I20260808 10:06:51.216976 20306 ts_tablet_manager.cc:1905] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:51.218771 20306 ts_tablet_manager.cc:1918] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:51.218962 20306 log.cc:1196] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/18a86884e3a245d7a3eb065b2513f233
I20260808 10:06:51.219369 20306 ts_tablet_manager.cc:1939] T 18a86884e3a245d7a3eb065b2513f233 P ce48ae530099403a90aedd4ed22d0798: Deleting consensus metadata
I20260808 10:06:51.219858 19560 catalog_manager.cc:4841] TS ce48ae530099403a90aedd4ed22d0798 (127.24.162.130:41007): tablet 18a86884e3a245d7a3eb065b2513f233 (table test-table [id=33f850b3f60c4fddb163c219de00de45]) successfully deleted
W20260808 10:06:51.229239 19564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:37193 (error 108)
I20260808 10:06:51.229527 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 19862
I20260808 10:06:51.231290 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:51.229607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:37193 (local address 127.0.0.1:45172)
0808 10:06:51.231199 (+ 1592us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:51.231250 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:37193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1568}
W20260808 10:06:51.233980 19564 catalog_manager.cc:4584] TS 505250bb16f64981a4120b61e158e469 (127.24.162.129:37193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18a86884e3a245d7a3eb065b2513f233: Network error: Client connection negotiation failed: client connection to 127.24.162.129:37193: connect: Connection refused (error 111)
I20260808 10:06:51.237823 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 20053
I20260808 10:06:51.245428 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 19539
2026-08-08T10:06:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:51 dist-test-slave-nfb5 krb5kdc[20311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:51 dist-test-slave-nfb5 krb5kdc[20311](info): set up 2 sockets
Aug 08 10:06:51 dist-test-slave-nfb5 krb5kdc[20311](info): commencing operation
krb5kdc: starting...
W20260808 10:06:52.473572 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:52 dist-test-slave-nfb5 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:06:52.601775 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33967
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:52.691213 20332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:52.691471 20332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:52.691510 20332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:52.692826 20332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:52.692911 20332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:52.692940 20332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:52.692974 20332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:52.693032 20332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:52.694641 20332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:52.694980 20332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:52.695238 20332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:52.700119 20338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:52.700129 20343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:52.700153 20339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:52.700572 20332 server_base.cc:1034] running on GCE node
I20260808 10:06:52.700839 20332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:52.701133 20332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:52.702312 20332 hybrid_clock.cc:648] HybridClock initialized: now 1786183612702305 us; error 37 us; skew 500 ppm
Aug 08 10:06:52 dist-test-slave-nfb5 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183612, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:52.704792 20332 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:06:52.705274 20332 webserver.cc:460] Webserver started at http://127.24.162.190:44009/ using document root <none> and password file <none>
I20260808 10:06:52.705509 20332 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:52.705577 20332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:52.705739 20332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:52.706833 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "a415e6ff6dd5cd278fb5b155d08f59dd"
server_key_iv: "8a64f033f73687e10de1ab19f24fafdf"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.707293 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "a415e6ff6dd5cd278fb5b155d08f59dd"
server_key_iv: "8a64f033f73687e10de1ab19f24fafdf"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.708750 20332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:06:52.709594 20354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:52.709800 20332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:52.709913 20332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "a415e6ff6dd5cd278fb5b155d08f59dd"
server_key_iv: "8a64f033f73687e10de1ab19f24fafdf"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.709991 20332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:52.723793 20332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:52.725790 20332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:52.725971 20332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:52.730463 20332 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33967
I20260808 10:06:52.730504 20430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33967 every 8 connection(s)
I20260808 10:06:52.730881 20332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:52.731842 20432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:52.734133 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: Bootstrap starting.
I20260808 10:06:52.734697 20432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:52.734966 20432 log.cc:824] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:52.735666 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: No bootstrap required, opened a new log
I20260808 10:06:52.737188 20432 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } }
I20260808 10:06:52.737340 20432 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:52.737365 20432 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f31ef0956a4a998b20e6f9cb1b7aab, State: Initialized, Role: FOLLOWER
I20260808 10:06:52.737509 20432 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } }
I20260808 10:06:52.737593 20432 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:52.737617 20432 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:52.737646 20432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:52.738065 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 20332
I20260808 10:06:52.738178 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:52.738191 20432 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } }
I20260808 10:06:52.738309 20432 leader_election.cc:302] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4f31ef0956a4a998b20e6f9cb1b7aab; no voters:
I20260808 10:06:52.738402 25226 external_mini_cluster.cc:1428] Setting key 8e3fccd547ffe70da59f9b7ffaa573f7
I20260808 10:06:52.738508 20432 leader_election.cc:288] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:52.738528 20435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:52.738651 20435 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [term 1 LEADER]: Becoming Leader. State: Replica: e4f31ef0956a4a998b20e6f9cb1b7aab, State: Running, Role: LEADER
I20260808 10:06:52.738744 20435 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } }
I20260808 10:06:52.738787 20432 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:52.739281 20436 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } } }
I20260808 10:06:52.739558 20436 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:52.739476 20437 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4f31ef0956a4a998b20e6f9cb1b7aab. Latest consensus state: current_term: 1 leader_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f31ef0956a4a998b20e6f9cb1b7aab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33967 } } }
I20260808 10:06:52.739847 20437 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:52.740326 20454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:52.741048 20454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:52.742679 20454 catalog_manager.cc:1347] Generated new cluster ID: be6bb5cffcac403381d9bf282d4bcbc4
I20260808 10:06:52.742745 20454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:06:52 dist-test-slave-nfb5 krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:52.749912 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:52.739156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.24.162.190:33967)
0808 10:06:52.739366 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:52.739377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:52.739458 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:06:52.739679 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:52.739686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:52.739746 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:52.739924 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:52.740878 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.742031 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.743078 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.743516 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.743633 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:52.746053 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:52.746057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:52.746058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:52.746070 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:52.748257 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:52.748655 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:52.748658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:52.748659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:52.748768 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:52.749000 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:52.749001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:52.749002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:52.749259 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:52.749440 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:52.749542 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:06:52.749689 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260808 10:06:52.772609 20454 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:52.773173 20454 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:52.777149 20454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: Generated new TSK 0
I20260808 10:06:52.777354 20454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.162.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
Entry for principal kudu/127.24.162.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab.
I20260808 10:06:52.811470 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:52.894289 20473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:52.894515 20473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:52.894539 20473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:52.895663 20473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:52.895709 20473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:52.895735 20473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:52.897362 20473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.129.keytab
--principal=kudu/127.24.162.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:52.897563 20473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:52.897768 20473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:52.902697 20481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:52.902729 20484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:52.902757 20482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:52.903069 20473 server_base.cc:1034] running on GCE node
I20260808 10:06:52.903227 20473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:52.903416 20473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:52.904548 20473 hybrid_clock.cc:648] HybridClock initialized: now 1786183612904539 us; error 28 us; skew 500 ppm
Aug 08 10:06:52 dist-test-slave-nfb5 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183612, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:52.907428 20473 init.cc:377] Logged in from keytab as kudu/127.24.162.129@KRBTEST.COM (short username kudu)
I20260808 10:06:52.907887 20473 webserver.cc:460] Webserver started at http://127.24.162.129:34199/ using document root <none> and password file <none>
I20260808 10:06:52.908133 20473 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:52.908179 20473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:52.908326 20473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:52.909384 20473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "bcc538b928494ee89e4fe7e934ff6044"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "de4c288bdfc705dd3e7bd7d0982084e9"
server_key_iv: "b7d9f6ab5735c7b4a5db9a1230938b1c"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.909745 20473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "bcc538b928494ee89e4fe7e934ff6044"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "de4c288bdfc705dd3e7bd7d0982084e9"
server_key_iv: "b7d9f6ab5735c7b4a5db9a1230938b1c"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.911152 20473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:52.911952 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:52.912158 20473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:52.912249 20473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "bcc538b928494ee89e4fe7e934ff6044"
format_stamp: "Formatted at 2026-08-08 10:06:52 on dist-test-slave-nfb5"
server_key: "de4c288bdfc705dd3e7bd7d0982084e9"
server_key_iv: "b7d9f6ab5735c7b4a5db9a1230938b1c"
server_key_version: "encryptionkey@0"
I20260808 10:06:52.912328 20473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:52.921507 20473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:52.923951 20473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:52.924065 20473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:52.924233 20473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:52.924542 20473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:52.924577 20473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:52.924620 20473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:52.924651 20473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:52.930671 20473 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:45561
I20260808 10:06:52.930723 20668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:45561 every 8 connection(s)
I20260808 10:06:52.931061 20473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:52.936780 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 20473
I20260808 10:06:52.936869 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:52.937060 25226 external_mini_cluster.cc:1428] Setting key f46602a1f5ed2ff71451fdfab20aaec3
Aug 08 10:06:52 dist-test-slave-nfb5 krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183612, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.129@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:52.940975 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:52.931844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:36867 (local address 127.24.162.190:33967)
0808 10:06:52.931895 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:52.931897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:52.932076 (+ 179us) server_negotiation.cc:407] Connection header received
0808 10:06:52.932551 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:52.932552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:52.932565 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:52.932628 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:52.933574 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.934103 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.935140 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.935331 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.935387 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:52.937890 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:52.937891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:52.937891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:52.937898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:52.939450 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:52.939930 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:52.939931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:52.939931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:52.939998 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:52.940274 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:52.940275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:52.940275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:52.940396 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:52.940479 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:52.940835 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 10:06:52.940910 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:06:52.941260 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:52.931759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33967 (local address 127.24.162.129:36867)
0808 10:06:52.931941 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:52.931985 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:52.932460 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:52.932708 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:52.932711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:52.933275 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:52.933483 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:52.933487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.934187 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.934188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:52.935065 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:52.935067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:52.935406 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:52.935407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:52.935513 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:52.936111 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:52.936120 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:52.937797 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:52.939540 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:52.939548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:52.939550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:52.939800 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:52.940081 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:52.940083 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:52.940083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:52.940201 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:52.940493 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:52.940501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:52.940683 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 10:06:52.940801 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 10:06:52.940979 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260808 10:06:52.941840 20669 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33967
I20260808 10:06:52.941928 20669 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:52.942137 20669 heartbeater.cc:511] Master 127.24.162.190:33967 requested a full tablet report, sending...
I20260808 10:06:52.942754 20379 ts_manager.cc:194] Registered new tserver with Master: bcc538b928494ee89e4fe7e934ff6044 (127.24.162.129:45561)
I20260808 10:06:52.943727 20379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.129@KRBTEST.COM'} at 127.24.162.129:36867
WARNING: no policy specified for kudu/127.24.162.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
Entry for principal kudu/127.24.162.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab.
I20260808 10:06:52.965548 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:53.051994 20679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:53.052194 20679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:53.052217 20679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:53.053349 20679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:53.053392 20679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:53.053419 20679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:53.054983 20679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.130.keytab
--principal=kudu/127.24.162.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:53.055258 20679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:53.055491 20679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:53.060761 20692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:53.060883 20679 server_base.cc:1034] running on GCE node
W20260808 10:06:53.060976 20695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.060770 20690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:53.061254 20679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:53.061511 20679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:53.062681 20679 hybrid_clock.cc:648] HybridClock initialized: now 1786183613062659 us; error 42 us; skew 500 ppm
Aug 08 10:06:53 dist-test-slave-nfb5 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:53.065358 20679 init.cc:377] Logged in from keytab as kudu/127.24.162.130@KRBTEST.COM (short username kudu)
I20260808 10:06:53.065798 20679 webserver.cc:460] Webserver started at http://127.24.162.130:36139/ using document root <none> and password file <none>
I20260808 10:06:53.066033 20679 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:53.066114 20679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:53.066277 20679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:53.067445 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "5f88ee42e94f4e06926e5646f789b72e"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "680be6b7066890f7a53236ba11440622"
server_key_iv: "eca65dfc090d77753f12aea3af8012f0"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.067943 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "5f88ee42e94f4e06926e5646f789b72e"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "680be6b7066890f7a53236ba11440622"
server_key_iv: "eca65dfc090d77753f12aea3af8012f0"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.069392 20679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:06:53.070186 20705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.070360 20679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:53.070464 20679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "5f88ee42e94f4e06926e5646f789b72e"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "680be6b7066890f7a53236ba11440622"
server_key_iv: "eca65dfc090d77753f12aea3af8012f0"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.070544 20679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:53.094880 20679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:53.098145 20679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:53.098281 20679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:53.098588 20679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:53.099048 20679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:53.099126 20679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.099169 20679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:53.099195 20679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.106444 20679 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:41227
I20260808 10:06:53.106488 20875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:41227 every 8 connection(s)
I20260808 10:06:53.106822 20679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:53.111039 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 20679
I20260808 10:06:53.111164 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:53.111353 25226 external_mini_cluster.cc:1428] Setting key 4221cc9d2c42badd8f181c903b6e2c08
Aug 08 10:06:53 dist-test-slave-nfb5 krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.130@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:53.119122 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.107627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:51031 (local address 127.24.162.190:33967)
0808 10:06:53.107708 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.107711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.107993 (+ 282us) server_negotiation.cc:407] Connection header received
0808 10:06:53.108795 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.108796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.108808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.108872 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:53.110262 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.110752 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.112433 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.112732 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.112819 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.115989 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:53.115990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:53.115991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:53.115998 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:53.117506 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.118036 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.118037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.118037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.118088 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.118401 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.118402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.118402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.118507 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:53.118556 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.118981 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.119057 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:06:53.119415 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.107593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33967 (local address 127.24.162.130:51031)
0808 10:06:53.107815 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.107868 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.108675 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.108955 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.108960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.109834 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:53.110146 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.110164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.110867 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.110868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.112299 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.112303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.112825 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.112827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.112995 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.113646 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:53.113659 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:53.115895 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:53.117600 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:53.117608 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:53.117610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:53.117957 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:53.118165 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:53.118166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:53.118166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:53.118324 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:53.118575 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:53.118583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:53.118777 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.118942 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.119157 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":117,"threads_started":1}
I20260808 10:06:53.119967 20876 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33967
I20260808 10:06:53.120069 20876 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:53.120262 20876 heartbeater.cc:511] Master 127.24.162.190:33967 requested a full tablet report, sending...
I20260808 10:06:53.120671 20379 ts_manager.cc:194] Registered new tserver with Master: 5f88ee42e94f4e06926e5646f789b72e (127.24.162.130:41227)
I20260808 10:06:53.121048 20379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.130@KRBTEST.COM'} at 127.24.162.130:51031
WARNING: no policy specified for kudu/127.24.162.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
Entry for principal kudu/127.24.162.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab.
I20260808 10:06:53.141769 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:53.230708 20882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:53.231011 20882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:53.231069 20882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:53.232570 20882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:53.232726 20882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:53.232791 20882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:53.234385 20882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.131.keytab
--principal=kudu/127.24.162.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:33967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:53.234687 20882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:53.234957 20882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:53.240109 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.241025 20894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.241220 20893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:53.241809 20882 server_base.cc:1034] running on GCE node
I20260808 10:06:53.242033 20882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:53.242296 20882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:53.243463 20882 hybrid_clock.cc:648] HybridClock initialized: now 1786183613243455 us; error 52 us; skew 500 ppm
Aug 08 10:06:53 dist-test-slave-nfb5 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:53.246193 20882 init.cc:377] Logged in from keytab as kudu/127.24.162.131@KRBTEST.COM (short username kudu)
I20260808 10:06:53.246750 20882 webserver.cc:460] Webserver started at http://127.24.162.131:37395/ using document root <none> and password file <none>
I20260808 10:06:53.247051 20882 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:53.247128 20882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:53.247283 20882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:53.248387 20882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "a42f8ecc28b7405f8d2c01ba51476514"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "d4f9ce04a94208b1e1873b996e910298"
server_key_iv: "c4bda93bbb1502200c93611f07a98e8d"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.248783 20882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "a42f8ecc28b7405f8d2c01ba51476514"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "d4f9ce04a94208b1e1873b996e910298"
server_key_iv: "c4bda93bbb1502200c93611f07a98e8d"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.250324 20882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:06:53.251196 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.251389 20882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:53.251492 20882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "a42f8ecc28b7405f8d2c01ba51476514"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "d4f9ce04a94208b1e1873b996e910298"
server_key_iv: "c4bda93bbb1502200c93611f07a98e8d"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.251580 20882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:53.264596 20882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:53.267102 20882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:53.267295 20882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:53.267550 20882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:53.267961 20882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:53.268085 20882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.268178 20882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:53.268213 20882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.274595 20882 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:38431
I20260808 10:06:53.274638 21094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:38431 every 8 connection(s)
I20260808 10:06:53.275108 20882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:53.277729 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 20882
I20260808 10:06:53.277815 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:53.277995 25226 external_mini_cluster.cc:1428] Setting key fed3e42e8368229bcbad11b344bb28b2
Aug 08 10:06:53 dist-test-slave-nfb5 krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.131@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:06:53.284752 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.276018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:58421 (local address 127.24.162.190:33967)
0808 10:06:53.276116 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.276118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.276214 (+ 96us) server_negotiation.cc:407] Connection header received
0808 10:06:53.276729 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.276730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.276742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.276799 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:53.277979 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.278543 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.279597 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.279816 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.279914 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.282028 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:53.282029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:53.282030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:53.282036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:53.283426 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.283819 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.283820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.283820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.283876 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.284103 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.284104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.284104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.284221 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:53.284274 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.284557 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.284636 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:06:53.284775 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.275921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:33967 (local address 127.24.162.131:58421)
0808 10:06:53.276084 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.276137 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.276636 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.276856 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.276859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.277574 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:53.277814 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.277821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.278643 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.278644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.279495 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.279497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.279926 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.279927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.280053 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.280524 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:06:53.280532 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:53.281943 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:53.283504 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:53.283511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:53.283512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:53.283736 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:53.283941 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:06:53.283942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:06:53.283942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:06:53.284044 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:06:53.284278 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:53.284282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:53.284401 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.284488 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.284608 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260808 10:06:53.285348 21095 heartbeater.cc:347] Connected to a master server at 127.24.162.190:33967
I20260808 10:06:53.285465 21095 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:53.285691 21095 heartbeater.cc:511] Master 127.24.162.190:33967 requested a full tablet report, sending...
I20260808 10:06:53.286123 20379 ts_manager.cc:194] Registered new tserver with Master: a42f8ecc28b7405f8d2c01ba51476514 (127.24.162.131:38431)
I20260808 10:06:53.286548 20379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.162.131@KRBTEST.COM'} at 127.24.162.131:58421
I20260808 10:06:53.289636 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:53.297556 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.291138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.24.162.190:33967)
0808 10:06:53.291207 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.291209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.291323 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:06:53.291363 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.291363 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.291372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.291413 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:53.291977 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.292442 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.293233 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.293414 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.293464 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.294797 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:53.294799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:53.294799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:53.294805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:53.296324 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.296675 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.296676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.296676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.296728 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:53.296909 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:53.296910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:53.296910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:53.297098 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:53.297175 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.297404 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.297483 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:06:53.299645 20379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:53.301349 20379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:53.308624 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.303933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:38431 (local address 127.0.0.1:41430)
0808 10:06:53.304310 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.304358 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.304429 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.305042 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.305042 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.305044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:53.305083 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.305084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.306077 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.306078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.307426 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.307429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.308294 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.308295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.308394 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.308411 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.308482 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.308539 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260808 10:06:53.308748 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.303968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45561 (local address 127.0.0.1:46848)
0808 10:06:53.304313 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.304358 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.304429 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.305363 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.305364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.305367 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:53.305401 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.305402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.306699 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.306700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.307721 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.307725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.308442 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.308443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.308516 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.308520 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.308584 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.308627 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":94,"threads_started":1}
I20260808 10:06:53.308996 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.304127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.24.162.131:38431)
0808 10:06:53.304872 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.304878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.304889 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:53.304928 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.304930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.304950 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.305034 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:53.305161 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.305990 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.307524 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.308201 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.308270 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.308310 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.308856 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.308900 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":80,"threads_started":1}
I20260808 10:06:53.309425 21113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.304121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.24.162.129:45561)
0808 10:06:53.305194 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.305199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.305210 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:53.305250 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.305252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.305267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.305353 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:53.305467 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.306583 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.307823 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.308340 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.308417 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.308473 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.308609 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.308682 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":80,"threads_started":1}
I20260808 10:06:53.309773 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.304187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:41227 (local address 127.0.0.1:42768)
0808 10:06:53.304337 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.304376 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.304435 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.304741 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.304744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.304746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:53.304824 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.304826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.306289 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.306290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.307430 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.307432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.309517 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.309518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.309589 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.309593 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.309649 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.309695 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":32,"thread_start_us":92,"threads_started":1}
I20260808 10:06:53.309733 20990 tablet_service.cc:1467] Processing CreateTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 (DEFAULT_TABLE table=test-table [id=36aec9c63cf3461e8eafdb0aa7453f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:53.310039 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.304314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.24.162.130:41227)
0808 10:06:53.304536 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.304542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.304553 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:53.304598 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.304600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.304617 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.304716 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:53.304908 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.306179 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.308344 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.309422 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.309503 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.309558 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.309902 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.309963 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
I20260808 10:06:53.310108 20990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e72e3ea81d44e939ea18c5ca36ccad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:53.310236 20572 tablet_service.cc:1467] Processing CreateTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 (DEFAULT_TABLE table=test-table [id=36aec9c63cf3461e8eafdb0aa7453f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:53.310539 20572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e72e3ea81d44e939ea18c5ca36ccad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:53.310634 20778 tablet_service.cc:1467] Processing CreateTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 (DEFAULT_TABLE table=test-table [id=36aec9c63cf3461e8eafdb0aa7453f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:53.310890 20778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e72e3ea81d44e939ea18c5ca36ccad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:53.313715 21118 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Bootstrap starting.
I20260808 10:06:53.313985 21120 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Bootstrap starting.
I20260808 10:06:53.314459 21119 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Bootstrap starting.
I20260808 10:06:53.314524 21118 tablet_bootstrap.cc:654] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:53.314677 21120 tablet_bootstrap.cc:654] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:53.315235 21118 log.cc:824] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:53.315246 21119 tablet_bootstrap.cc:654] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:53.315307 21120 log.cc:824] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:53.315910 21119 log.cc:824] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:53.316139 21120 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: No bootstrap required, opened a new log
I20260808 10:06:53.316210 21120 ts_tablet_manager.cc:1397] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:53.316438 21118 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: No bootstrap required, opened a new log
I20260808 10:06:53.316532 21118 ts_tablet_manager.cc:1397] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 10:06:53.317605 21120 raft_consensus.cc:348] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.317749 21120 raft_consensus.cc:374] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:53.317771 21120 raft_consensus.cc:729] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f88ee42e94f4e06926e5646f789b72e, State: Initialized, Role: FOLLOWER
I20260808 10:06:53.317971 21120 consensus_queue.cc:260] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.318135 21118 raft_consensus.cc:348] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.318262 21120 ts_tablet_manager.cc:1428] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:53.318336 20876 heartbeater.cc:503] Master 127.24.162.190:33967 was elected leader, sending a full tablet report...
I20260808 10:06:53.318326 21118 raft_consensus.cc:374] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:53.318380 21118 raft_consensus.cc:729] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42f8ecc28b7405f8d2c01ba51476514, State: Initialized, Role: FOLLOWER
I20260808 10:06:53.318502 21118 consensus_queue.cc:260] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.318833 21095 heartbeater.cc:503] Master 127.24.162.190:33967 was elected leader, sending a full tablet report...
I20260808 10:06:53.319015 21119 tablet_bootstrap.cc:492] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: No bootstrap required, opened a new log
I20260808 10:06:53.319139 21118 ts_tablet_manager.cc:1428] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:06:53.319237 21119 ts_tablet_manager.cc:1397] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 10:06:53.320900 21119 raft_consensus.cc:348] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.321026 21119 raft_consensus.cc:374] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:53.321048 21119 raft_consensus.cc:729] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc538b928494ee89e4fe7e934ff6044, State: Initialized, Role: FOLLOWER
I20260808 10:06:53.321151 21119 consensus_queue.cc:260] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.321431 21119 ts_tablet_manager.cc:1428] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:53.321455 20669 heartbeater.cc:503] Master 127.24.162.190:33967 was elected leader, sending a full tablet report...
W20260808 10:06:53.357502 20877 tablet.cc:2369] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:06:53.431797 20674 tablet.cc:2369] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:53.495957 21125 raft_consensus.cc:482] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:53.496241 21125 raft_consensus.cc:504] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.496658 21125 leader_election.cc:288] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a42f8ecc28b7405f8d2c01ba51476514 (127.24.162.131:38431), bcc538b928494ee89e4fe7e934ff6044 (127.24.162.129:45561)
I20260808 10:06:53.499691 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.496907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45561 (local address 127.24.162.130:44013)
0808 10:06:53.496979 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.496993 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.497041 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.497281 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.497282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.497283 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:53.497359 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.497361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.498036 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.498037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.498808 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.498810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.499411 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.499412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.499496 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.499500 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.499555 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.499599 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:06:53.499715 21113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.497039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:44013 (local address 127.24.162.129:45561)
0808 10:06:53.497096 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.497098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.497106 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:06:53.497141 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.497142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.497151 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.497256 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:53.497444 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.497947 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.498928 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.499331 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.499388 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.499395 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.499576 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.499618 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:06:53.499853 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.497005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:53611 (local address 127.24.162.131:38431)
0808 10:06:53.497059 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.497061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.497214 (+ 153us) server_negotiation.cc:407] Connection header received
0808 10:06:53.497247 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.497249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.497259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.497309 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:53.497533 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.498050 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.499017 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.499420 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.499495 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.499503 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.499728 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.499773 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:06:53.500064 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.496906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:38431 (local address 127.24.162.130:53611)
0808 10:06:53.497105 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.497114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.497200 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.497340 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.497341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.497343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:53.497449 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.497451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.498144 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.498145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.498914 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.498917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.499507 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.499508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.499616 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.499620 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.499947 (+ 327us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.499992 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":16,"thread_start_us":104,"threads_started":1}
I20260808 10:06:53.500221 20602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e72e3ea81d44e939ea18c5ca36ccad3" candidate_uuid: "5f88ee42e94f4e06926e5646f789b72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcc538b928494ee89e4fe7e934ff6044" is_pre_election: true
I20260808 10:06:53.500433 20602 raft_consensus.cc:2454] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f88ee42e94f4e06926e5646f789b72e in term 0.
I20260808 10:06:53.500476 21026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e72e3ea81d44e939ea18c5ca36ccad3" candidate_uuid: "5f88ee42e94f4e06926e5646f789b72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42f8ecc28b7405f8d2c01ba51476514" is_pre_election: true
I20260808 10:06:53.500674 21026 raft_consensus.cc:2454] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f88ee42e94f4e06926e5646f789b72e in term 0.
I20260808 10:06:53.500744 20707 leader_election.cc:302] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f88ee42e94f4e06926e5646f789b72e, bcc538b928494ee89e4fe7e934ff6044; no voters:
I20260808 10:06:53.500871 21125 raft_consensus.cc:2781] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:53.500927 21125 raft_consensus.cc:482] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:53.500950 21125 raft_consensus.cc:3037] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:53.501683 21125 raft_consensus.cc:504] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.501838 21125 leader_election.cc:288] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [CANDIDATE]: Term 1 election: Requested vote from peers a42f8ecc28b7405f8d2c01ba51476514 (127.24.162.131:38431), bcc538b928494ee89e4fe7e934ff6044 (127.24.162.129:45561)
I20260808 10:06:53.501998 20602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e72e3ea81d44e939ea18c5ca36ccad3" candidate_uuid: "5f88ee42e94f4e06926e5646f789b72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcc538b928494ee89e4fe7e934ff6044"
I20260808 10:06:53.502020 21026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e72e3ea81d44e939ea18c5ca36ccad3" candidate_uuid: "5f88ee42e94f4e06926e5646f789b72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42f8ecc28b7405f8d2c01ba51476514"
I20260808 10:06:53.502100 20602 raft_consensus.cc:3037] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:53.502096 21026 raft_consensus.cc:3037] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:53.502822 21026 raft_consensus.cc:2454] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f88ee42e94f4e06926e5646f789b72e in term 1.
I20260808 10:06:53.503019 20602 raft_consensus.cc:2454] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f88ee42e94f4e06926e5646f789b72e in term 1.
I20260808 10:06:53.503032 20706 leader_election.cc:302] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f88ee42e94f4e06926e5646f789b72e, a42f8ecc28b7405f8d2c01ba51476514; no voters:
I20260808 10:06:53.503183 21125 raft_consensus.cc:2781] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:53.503372 21125 raft_consensus.cc:686] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 1 LEADER]: Becoming Leader. State: Replica: 5f88ee42e94f4e06926e5646f789b72e, State: Running, Role: LEADER
I20260808 10:06:53.503476 21125 consensus_queue.cc:237] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } }
I20260808 10:06:53.504341 20378 catalog_manager.cc:5495] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f88ee42e94f4e06926e5646f789b72e (127.24.162.130). New cstate: current_term: 1 leader_uuid: "5f88ee42e94f4e06926e5646f789b72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f88ee42e94f4e06926e5646f789b72e" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 41227 } health_report { overall_health: HEALTHY } } }
W20260808 10:06:53.525736 21096 tablet.cc:2369] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:53.535372 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.532799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.24.162.130:41227)
0808 10:06:53.532866 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.532868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.532879 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:06:53.532918 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.532918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.532937 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.533002 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:53.533279 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.533803 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.534565 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.534769 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.534835 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.534989 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:53.535093 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:53.535159 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.535194 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.535215 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:06:53.537479 21026 raft_consensus.cc:1261] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 1 FOLLOWER]: Refusing update from remote peer 5f88ee42e94f4e06926e5646f789b72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:53.537493 20602 raft_consensus.cc:1261] T 3e72e3ea81d44e939ea18c5ca36ccad3 P bcc538b928494ee89e4fe7e934ff6044 [term 1 FOLLOWER]: Refusing update from remote peer 5f88ee42e94f4e06926e5646f789b72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:53.537906 21135 consensus_queue.cc:1035] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42f8ecc28b7405f8d2c01ba51476514" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 38431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:53.538051 21125 consensus_queue.cc:1035] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcc538b928494ee89e4fe7e934ff6044" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 45561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:53.541015 21139 mvcc.cc:205] Tried to move back new op lower bound from 7316208081046437888 to 7316208080910708736. Current Snapshot: MvccSnapshot[applied={T|T < 7316208081046437888}]
I20260808 10:06:53.541175 21140 mvcc.cc:205] Tried to move back new op lower bound from 7316208081046437888 to 7316208080910708736. Current Snapshot: MvccSnapshot[applied={T|T < 7316208081046437888}]
I20260808 10:06:53.543556 20378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:53.543663 20378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:53.544502 20378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4f31ef0956a4a998b20e6f9cb1b7aab: Sending DeleteTablet for 3 replicas of tablet 3e72e3ea81d44e939ea18c5ca36ccad3
I20260808 10:06:53.544912 20990 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:41430
I20260808 10:06:53.544946 20572 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:46848
I20260808 10:06:53.545173 20778 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e72e3ea81d44e939ea18c5ca36ccad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:53 UTC) from {username='kudu', principal='kudu/127.24.162.190@KRBTEST.COM'} at 127.0.0.1:42768
I20260808 10:06:53.545418 21153 tablet_replica.cc:330] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: stopping tablet replica
I20260808 10:06:53.545499 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 20473
I20260808 10:06:53.545524 21153 raft_consensus.cc:2229] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:53.559072 21154 tablet_replica.cc:330] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: stopping tablet replica
I20260808 10:06:53.559242 21154 raft_consensus.cc:2229] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:53.559405 21154 raft_consensus.cc:2258] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:53.560549 21154 ts_tablet_manager.cc:1905] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:53.562346 21154 ts_tablet_manager.cc:1918] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:53.562453 21154 log.cc:1196] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/3e72e3ea81d44e939ea18c5ca36ccad3
I20260808 10:06:53.562744 21154 ts_tablet_manager.cc:1939] T 3e72e3ea81d44e939ea18c5ca36ccad3 P 5f88ee42e94f4e06926e5646f789b72e: Deleting consensus metadata
I20260808 10:06:53.563251 20355 catalog_manager.cc:4841] TS 5f88ee42e94f4e06926e5646f789b72e (127.24.162.130:41227): tablet 3e72e3ea81d44e939ea18c5ca36ccad3 (table test-table [id=36aec9c63cf3461e8eafdb0aa7453f36]) successfully deleted
I20260808 10:06:53.563411 21153 raft_consensus.cc:2258] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:53.563980 21153 ts_tablet_manager.cc:1905] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:53.565981 21153 ts_tablet_manager.cc:1918] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:53.566069 21153 log.cc:1196] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/3e72e3ea81d44e939ea18c5ca36ccad3
I20260808 10:06:53.566367 21153 ts_tablet_manager.cc:1939] T 3e72e3ea81d44e939ea18c5ca36ccad3 P a42f8ecc28b7405f8d2c01ba51476514: Deleting consensus metadata
I20260808 10:06:53.566947 20355 catalog_manager.cc:4841] TS a42f8ecc28b7405f8d2c01ba51476514 (127.24.162.131:38431): tablet 3e72e3ea81d44e939ea18c5ca36ccad3 (table test-table [id=36aec9c63cf3461e8eafdb0aa7453f36]) successfully deleted
W20260808 10:06:53.570281 20356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:45561 (error 108)
I20260808 10:06:53.570461 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 20679
I20260808 10:06:53.571307 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.571131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:45561 (local address 127.0.0.1:46856)
0808 10:06:53.571198 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.571261 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:45561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:06:53.572340 20356 catalog_manager.cc:4584] TS bcc538b928494ee89e4fe7e934ff6044 (127.24.162.129:45561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e72e3ea81d44e939ea18c5ca36ccad3: Network error: Client connection negotiation failed: client connection to 127.24.162.129:45561: connect: Connection refused (error 111)
I20260808 10:06:53.581096 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 20882
I20260808 10:06:53.591637 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 20332
2026-08-08T10:06:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2348 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T10:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:53Z Disabled control of system clock
I20260808 10:06:53.624790 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37933
--webserver_interface=127.24.162.190
--webserver_port=0
--builtin_ntp_servers=127.24.162.148:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:37933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:53.708369 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:53.708592 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:53.708614 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:53.709865 21162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:53.709945 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:53.709970 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:53.709990 21162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:53.710019 21162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:53.711463 21162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:37933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:53.711735 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:53.712045 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:53.717109 21172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.717126 21171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.717140 21176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:53.717341 21162 server_base.cc:1034] running on GCE node
I20260808 10:06:53.717568 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:53.717893 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:53.719028 21162 hybrid_clock.cc:648] HybridClock initialized: now 1786183613719009 us; error 41 us; skew 500 ppm
I20260808 10:06:53.720304 21162 webserver.cc:460] Webserver started at http://127.24.162.190:44509/ using document root <none> and password file <none>
I20260808 10:06:53.720573 21162 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:53.720664 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:53.720870 21162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:53.721969 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "bc927c803ce84211b04ddccd5f8bb951"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "0b32a48b3c7e2fe05921dd64783a1d62"
server_key_iv: "5b6e851bfcbe8ffefb49e8ea90d2ec02"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.722352 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "bc927c803ce84211b04ddccd5f8bb951"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "0b32a48b3c7e2fe05921dd64783a1d62"
server_key_iv: "5b6e851bfcbe8ffefb49e8ea90d2ec02"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.723824 21162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:06:53.724581 21186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.724779 21162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:53.724876 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "bc927c803ce84211b04ddccd5f8bb951"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "0b32a48b3c7e2fe05921dd64783a1d62"
server_key_iv: "5b6e851bfcbe8ffefb49e8ea90d2ec02"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.724960 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:53.738226 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:53.738612 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:53.738783 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:53.743357 21162 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:37933
I20260808 10:06:53.743404 21254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:37933 every 8 connection(s)
I20260808 10:06:53.743782 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:53.744802 21255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:53.747207 21255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: Bootstrap starting.
I20260808 10:06:53.747804 21255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:53.748114 21255 log.cc:824] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:53.748847 21255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: No bootstrap required, opened a new log
I20260808 10:06:53.749683 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 21162
I20260808 10:06:53.749786 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:53.749984 25226 external_mini_cluster.cc:1428] Setting key 21188ea1165405ca730bf74e52103748
I20260808 10:06:53.751114 21255 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } }
I20260808 10:06:53.751281 21255 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:53.751349 21255 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc927c803ce84211b04ddccd5f8bb951, State: Initialized, Role: FOLLOWER
I20260808 10:06:53.751523 21255 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } }
I20260808 10:06:53.751619 21255 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:53.751652 21255 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:53.751731 21255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:53.752486 21255 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } }
I20260808 10:06:53.752600 21255 leader_election.cc:302] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc927c803ce84211b04ddccd5f8bb951; no voters:
I20260808 10:06:53.752750 21255 leader_election.cc:288] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:53.752841 21263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:53.752998 21255 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:53.753079 21263 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [term 1 LEADER]: Becoming Leader. State: Replica: bc927c803ce84211b04ddccd5f8bb951, State: Running, Role: LEADER
I20260808 10:06:53.753216 21263 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } }
I20260808 10:06:53.753633 21264 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc927c803ce84211b04ddccd5f8bb951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } } }
I20260808 10:06:53.753644 21265 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc927c803ce84211b04ddccd5f8bb951. Latest consensus state: current_term: 1 leader_uuid: "bc927c803ce84211b04ddccd5f8bb951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc927c803ce84211b04ddccd5f8bb951" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37933 } } }
I20260808 10:06:53.753760 21264 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:53.753764 21265 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951 [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:53.753968 21276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:53.754963 21276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:53.757145 21276 catalog_manager.cc:1347] Generated new cluster ID: 229a17670ed341908f0d1c7d7ac941b8
I20260808 10:06:53.757215 21276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:53.757388 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.750602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.24.162.190:37933)
0808 10:06:53.750866 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.750876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.750942 (+ 66us) server_negotiation.cc:407] Connection header received
0808 10:06:53.751068 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.751073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.751128 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.751270 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:53.751512 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.753322 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.754530 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.755231 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.755361 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.756370 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:53.756374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:53.756375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:53.756583 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:53.756625 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:53.756646 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:53.756720 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.756961 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.757060 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260808 10:06:53.777524 21276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:53.778079 21276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:53.781690 21276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: Generated new TSK 0
I20260808 10:06:53.781878 21276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:53.785284 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.129:0
--local_ip_for_outbound_sockets=127.24.162.129
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--builtin_ntp_servers=127.24.162.148:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:53.872471 21291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:53.872720 21291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:53.872758 21291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:53.873890 21291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:53.873958 21291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:53.874006 21291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.129
I20260808 10:06:53.875442 21291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.162.129
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.129
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:53.875722 21291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:53.876036 21291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:53.882097 21298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.882097 21299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:53.882210 21301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:53.882402 21291 server_base.cc:1034] running on GCE node
I20260808 10:06:53.882596 21291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:53.882905 21291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:53.884124 21291 hybrid_clock.cc:648] HybridClock initialized: now 1786183613884102 us; error 38 us; skew 500 ppm
I20260808 10:06:53.885511 21291 webserver.cc:460] Webserver started at http://127.24.162.129:42157/ using document root <none> and password file <none>
I20260808 10:06:53.885802 21291 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:53.885876 21291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:53.886034 21291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:53.887229 21291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data/instance:
uuid: "3ac98d46427a45b5a3e958df45098f20"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "133284faaa69b8cafdd89f60c9d65c23"
server_key_iv: "e8f7fd2a20c52398cc0f1948e1393996"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.887601 21291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance:
uuid: "3ac98d46427a45b5a3e958df45098f20"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "133284faaa69b8cafdd89f60c9d65c23"
server_key_iv: "e8f7fd2a20c52398cc0f1948e1393996"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.888921 21291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:06:53.889691 21314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.889878 21291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:06:53.889986 21291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal
uuid: "3ac98d46427a45b5a3e958df45098f20"
format_stamp: "Formatted at 2026-08-08 10:06:53 on dist-test-slave-nfb5"
server_key: "133284faaa69b8cafdd89f60c9d65c23"
server_key_iv: "e8f7fd2a20c52398cc0f1948e1393996"
server_key_version: "encryptionkey@0"
I20260808 10:06:53.890064 21291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:53.912866 21291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:53.913228 21291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:53.913390 21291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:53.913632 21291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:53.913992 21291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:53.914052 21291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.914095 21291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:53.914119 21291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:53.920234 21291 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.129:41469
I20260808 10:06:53.920280 21488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.129:41469 every 8 connection(s)
I20260808 10:06:53.920621 21291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/data/info.pb
I20260808 10:06:53.921563 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 21291
I20260808 10:06:53.921690 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-0/wal/instance
I20260808 10:06:53.921904 25226 external_mini_cluster.cc:1428] Setting key 3918aed0804392e0d7f2b54ae3fc7609
I20260808 10:06:53.923084 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.130:0
--local_ip_for_outbound_sockets=127.24.162.130
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--builtin_ntp_servers=127.24.162.148:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:06:53.927062 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.921397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.129:48083 (local address 127.24.162.190:37933)
0808 10:06:53.921478 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:53.921481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:53.921788 (+ 307us) server_negotiation.cc:407] Connection header received
0808 10:06:53.922508 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:53.922509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:53.922524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:53.922585 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:53.923621 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.924120 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.925410 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.925740 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.925833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.926726 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:53.926727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:53.926727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:53.926736 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:53.926754 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:53.926759 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:53.926827 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:53.926888 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:06:53.926940 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:06:53.927253 21493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:53.921415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37933 (local address 127.24.162.129:48083)
0808 10:06:53.921637 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:53.921692 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:53.922399 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:53.922663 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:53.922667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:53.922678 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:53.923485 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.923493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.924221 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.924222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.925308 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:53.925311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:53.925841 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:53.925842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:53.925969 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:53.926574 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:53.926574 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:53.926602 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:53.926625 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:53.926825 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:53.926830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:53.926832 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:06:53.926883 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:06:53.927027 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260808 10:06:53.927796 21490 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37933
I20260808 10:06:53.927879 21490 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:53.928124 21490 heartbeater.cc:511] Master 127.24.162.190:37933 requested a full tablet report, sending...
I20260808 10:06:53.928735 21207 ts_manager.cc:194] Registered new tserver with Master: 3ac98d46427a45b5a3e958df45098f20 (127.24.162.129:41469)
I20260808 10:06:53.929729 21207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.129:48083
W20260808 10:06:54.007411 21495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:54.007645 21495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:54.007681 21495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:54.008816 21495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:54.008888 21495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:54.008931 21495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.130
I20260808 10:06:54.010272 21495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.162.130
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.130
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:54.010577 21495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:54.010906 21495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:54.016156 21503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:54.016252 21495 server_base.cc:1034] running on GCE node
W20260808 10:06:54.016235 21507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:54.016235 21505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:54.016579 21495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:54.016832 21495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:54.018002 21495 hybrid_clock.cc:648] HybridClock initialized: now 1786183614017992 us; error 46 us; skew 500 ppm
I20260808 10:06:54.019357 21495 webserver.cc:460] Webserver started at http://127.24.162.130:35749/ using document root <none> and password file <none>
I20260808 10:06:54.019656 21495 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:54.019739 21495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:54.019882 21495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:54.020975 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data/instance:
uuid: "f1cac21230db4e419016e8867a32b9fe"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "5f73e2691f09bbc3f8056b706746e084"
server_key_iv: "1b1893e18ec694c1cd7aae764da17285"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.021359 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance:
uuid: "f1cac21230db4e419016e8867a32b9fe"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "5f73e2691f09bbc3f8056b706746e084"
server_key_iv: "1b1893e18ec694c1cd7aae764da17285"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.022692 21495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:06:54.023586 21517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.023800 21495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:06:54.023907 21495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal
uuid: "f1cac21230db4e419016e8867a32b9fe"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "5f73e2691f09bbc3f8056b706746e084"
server_key_iv: "1b1893e18ec694c1cd7aae764da17285"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.023995 21495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:54.034022 21495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:54.034332 21495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:54.034487 21495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:54.034713 21495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:54.035099 21495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:54.035157 21495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.035197 21495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:54.035226 21495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.041435 21495 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.130:37399
I20260808 10:06:54.041510 21684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.130:37399 every 8 connection(s)
I20260808 10:06:54.041879 21495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/data/info.pb
I20260808 10:06:54.047538 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.042709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.130:53173 (local address 127.24.162.190:37933)
0808 10:06:54.042772 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.042774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.043134 (+ 360us) server_negotiation.cc:407] Connection header received
0808 10:06:54.043687 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.043688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.043701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.043761 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:54.044470 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.044980 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.045998 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.046217 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.046311 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.047228 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:54.047229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:54.047229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:54.047235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:54.047252 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:54.047257 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:54.047314 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.047421 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.047452 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:06:54.047760 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.042755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37933 (local address 127.24.162.130:53173)
0808 10:06:54.042999 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.043041 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.043569 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.043831 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.043835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.043845 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:54.044367 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.044371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.045093 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.045094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.045891 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.045894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.046330 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.046331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.046469 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.047078 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:54.047079 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:54.047108 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:54.047130 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:54.047325 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:54.047331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:54.047334 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.047422 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.047528 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260808 10:06:54.048295 21685 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37933
I20260808 10:06:54.048383 21685 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:54.048530 21685 heartbeater.cc:511] Master 127.24.162.190:37933 requested a full tablet report, sending...
I20260808 10:06:54.048936 21207 ts_manager.cc:194] Registered new tserver with Master: f1cac21230db4e419016e8867a32b9fe (127.24.162.130:37399)
I20260808 10:06:54.049273 21207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.130:53173
I20260808 10:06:54.049429 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 21495
I20260808 10:06:54.049552 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal/instance
I20260808 10:06:54.049753 25226 external_mini_cluster.cc:1428] Setting key 7559c843352391e9d22f415a4d6ccaae
I20260808 10:06:54.050711 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.162.131:0
--local_ip_for_outbound_sockets=127.24.162.131
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--builtin_ntp_servers=127.24.162.148:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:06:54.132915 21688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:54.133114 21688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:54.133136 21688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:54.134256 21688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:54.134300 21688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:54.134362 21688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.162.131
I20260808 10:06:54.135735 21688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.162.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.162.131
--webserver_port=0
--tserver_master_addrs=127.24.162.190:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.162.131
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:54.135996 21688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:54.136299 21688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:54.141388 21700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:54.141419 21698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:54.141395 21697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:54.141837 21688 server_base.cc:1034] running on GCE node
I20260808 10:06:54.142036 21688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:54.142271 21688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:54.143411 21688 hybrid_clock.cc:648] HybridClock initialized: now 1786183614143392 us; error 40 us; skew 500 ppm
I20260808 10:06:54.144754 21688 webserver.cc:460] Webserver started at http://127.24.162.131:34859/ using document root <none> and password file <none>
I20260808 10:06:54.145006 21688 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:54.145056 21688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:54.145210 21688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:54.146283 21688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data/instance:
uuid: "93bc9fdec7b740f4bc031562ba00ac50"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "8329ad12ef28cff2b392972edebd0ef3"
server_key_iv: "db4ce2e56f2e5b583f093eb52c9be28c"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.146657 21688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance:
uuid: "93bc9fdec7b740f4bc031562ba00ac50"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "8329ad12ef28cff2b392972edebd0ef3"
server_key_iv: "db4ce2e56f2e5b583f093eb52c9be28c"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.148056 21688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:06:54.148818 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.149008 21688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:54.149111 21688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal
uuid: "93bc9fdec7b740f4bc031562ba00ac50"
format_stamp: "Formatted at 2026-08-08 10:06:54 on dist-test-slave-nfb5"
server_key: "8329ad12ef28cff2b392972edebd0ef3"
server_key_iv: "db4ce2e56f2e5b583f093eb52c9be28c"
server_key_version: "encryptionkey@0"
I20260808 10:06:54.149199 21688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:54.164078 21688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:54.164392 21688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:54.164551 21688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:54.164791 21688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:06:54.165126 21688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:06:54.165182 21688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.165225 21688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:06:54.165254 21688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:54.171131 21688 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.131:46553
I20260808 10:06:54.171192 21886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.131:46553 every 8 connection(s)
I20260808 10:06:54.171506 21688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/data/info.pb
I20260808 10:06:54.176272 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 21688
I20260808 10:06:54.176404 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal/instance
I20260808 10:06:54.176609 25226 external_mini_cluster.cc:1428] Setting key a9038738c502e5d899b8bd04f49724d9
I20260808 10:06:54.176982 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.172241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:43687 (local address 127.24.162.190:37933)
0808 10:06:54.172306 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.172308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.172543 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:06:54.173067 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.173069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.173083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.173158 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:54.173867 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.174382 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.175373 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.175586 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.175676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.176649 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:54.176651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:54.176651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:54.176656 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:54.176671 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:54.176675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:54.176728 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.176860 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.176891 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:06:54.177277 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.172249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:37933 (local address 127.24.162.131:43687)
0808 10:06:54.172415 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.172450 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.172955 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.173230 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.173234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.173242 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:06:54.173753 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.173757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.174487 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.174489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.175253 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.175256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.175685 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.175686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.175817 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.176490 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:06:54.176491 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:06:54.176533 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:06:54.176556 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:06:54.176739 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:06:54.176747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:06:54.176751 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.176839 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.176921 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260808 10:06:54.177697 21887 heartbeater.cc:347] Connected to a master server at 127.24.162.190:37933
I20260808 10:06:54.177799 21887 heartbeater.cc:464] Registering TS with master...
I20260808 10:06:54.177984 21887 heartbeater.cc:511] Master 127.24.162.190:37933 requested a full tablet report, sending...
I20260808 10:06:54.178380 21207 ts_manager.cc:194] Registered new tserver with Master: 93bc9fdec7b740f4bc031562ba00ac50 (127.24.162.131:46553)
I20260808 10:06:54.178704 21207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.162.131:43687
I20260808 10:06:54.188486 25226 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:06:54.192656 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.189883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.24.162.190:37933)
0808 10:06:54.189971 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.189973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.190012 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:06:54.190063 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.190064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.190075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.190121 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:54.190271 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.190746 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.191526 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.191704 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.191756 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.192416 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:54.192417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:54.192417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:54.192421 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:54.192434 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:54.192437 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:54.192480 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.192514 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.192535 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:06:54.194491 21207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:06:54.196029 21207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:06:54.202831 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.24.162.130:37399)
0808 10:06:54.198757 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.198762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.198774 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:06:54.198819 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.198821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.198838 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.198989 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:54.199178 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200087 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.201660 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202361 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202441 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.202531 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.202676 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.202719 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":98,"threads_started":1}
I20260808 10:06:54.202816 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:37399 (local address 127.0.0.1:59546)
0808 10:06:54.198542 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.198589 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.198660 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.198994 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.198997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.198999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:54.199075 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.199077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200196 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.200198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.201551 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.201554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202464 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.202550 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.202564 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.202646 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.202719 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":64,"threads_started":1}
I20260808 10:06:54.203001 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41469 (local address 127.0.0.1:32822)
0808 10:06:54.198804 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.198815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.198865 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.199325 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.199326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.199329 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:54.199370 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.199372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200624 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.200625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.201658 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.201661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202647 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.202707 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.202710 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.202763 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.202798 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":59,"thread_start_us":126,"threads_started":1}
I20260808 10:06:54.203214 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.131:46553 (local address 127.0.0.1:34234)
0808 10:06:54.198818 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.198830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.198874 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.199264 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.199264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.199267 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:54.199312 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.199314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200612 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.200614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.201922 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.201926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202974 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.203040 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.203045 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.203097 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.203143 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":28,"thread_start_us":107,"threads_started":1}
I20260808 10:06:54.203431 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.24.162.131:46553)
0808 10:06:54.199123 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.199127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.199136 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:06:54.199171 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.199172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.199187 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.199261 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:54.199381 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200505 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202027 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202752 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202833 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.202933 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.203314 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.203355 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":99,"threads_started":1}
I20260808 10:06:54.203701 21590 tablet_service.cc:1467] Processing CreateTablet for tablet a629c05239654b9fb81c15f399a65199 (DEFAULT_TABLE table=test-table [id=2e74d57174d842b88f5d9750a71015b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:54.203848 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.198541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.24.162.129:41469)
0808 10:06:54.199000 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.199005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.199015 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:54.199056 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.199058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.199073 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.199133 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:54.199439 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.200533 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.201765 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.202400 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.202511 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.202609 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.202863 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.203756 (+ 893us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":84,"threads_started":1}
I20260808 10:06:54.204116 21590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a629c05239654b9fb81c15f399a65199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:54.204128 21793 tablet_service.cc:1467] Processing CreateTablet for tablet a629c05239654b9fb81c15f399a65199 (DEFAULT_TABLE table=test-table [id=2e74d57174d842b88f5d9750a71015b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:54.204493 21793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a629c05239654b9fb81c15f399a65199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:54.204471 21380 tablet_service.cc:1467] Processing CreateTablet for tablet a629c05239654b9fb81c15f399a65199 (DEFAULT_TABLE table=test-table [id=2e74d57174d842b88f5d9750a71015b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:06:54.204798 21380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a629c05239654b9fb81c15f399a65199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:54.207587 21909 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Bootstrap starting.
I20260808 10:06:54.207913 21910 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Bootstrap starting.
I20260808 10:06:54.208375 21909 tablet_bootstrap.cc:654] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:54.208571 21910 tablet_bootstrap.cc:654] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:54.208772 21909 log.cc:824] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:54.208885 21910 log.cc:824] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:54.209260 21911 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: Bootstrap starting.
I20260808 10:06:54.209585 21909 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: No bootstrap required, opened a new log
I20260808 10:06:54.209609 21910 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: No bootstrap required, opened a new log
I20260808 10:06:54.209664 21910 ts_tablet_manager.cc:1397] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:06:54.209685 21909 ts_tablet_manager.cc:1397] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:06:54.211619 21911 tablet_bootstrap.cc:654] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:54.211957 21911 log.cc:824] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:54.211917 21910 raft_consensus.cc:348] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.212071 21910 raft_consensus.cc:374] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:54.212124 21910 raft_consensus.cc:729] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cac21230db4e419016e8867a32b9fe, State: Initialized, Role: FOLLOWER
I20260808 10:06:54.212088 21909 raft_consensus.cc:348] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.212252 21910 consensus_queue.cc:260] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.212270 21909 raft_consensus.cc:374] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:54.212313 21909 raft_consensus.cc:729] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93bc9fdec7b740f4bc031562ba00ac50, State: Initialized, Role: FOLLOWER
I20260808 10:06:54.212502 21909 consensus_queue.cc:260] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.212539 21685 heartbeater.cc:503] Master 127.24.162.190:37933 was elected leader, sending a full tablet report...
I20260808 10:06:54.212522 21910 ts_tablet_manager.cc:1428] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:54.212817 21909 ts_tablet_manager.cc:1428] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:06:54.212838 21887 heartbeater.cc:503] Master 127.24.162.190:37933 was elected leader, sending a full tablet report...
I20260808 10:06:54.213548 21911 tablet_bootstrap.cc:492] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: No bootstrap required, opened a new log
I20260808 10:06:54.213621 21911 ts_tablet_manager.cc:1397] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:06:54.215499 21911 raft_consensus.cc:348] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.215682 21911 raft_consensus.cc:374] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:54.215708 21911 raft_consensus.cc:729] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac98d46427a45b5a3e958df45098f20, State: Initialized, Role: FOLLOWER
I20260808 10:06:54.215829 21911 consensus_queue.cc:260] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.216091 21911 ts_tablet_manager.cc:1428] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:06:54.216162 21490 heartbeater.cc:503] Master 127.24.162.190:37933 was elected leader, sending a full tablet report...
W20260808 10:06:54.292563 21686 tablet.cc:2369] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:06:54.295074 21916 raft_consensus.cc:482] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:06:54.295231 21916 raft_consensus.cc:504] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.295662 21916 leader_election.cc:288] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1cac21230db4e419016e8867a32b9fe (127.24.162.130:37399), 3ac98d46427a45b5a3e958df45098f20 (127.24.162.129:41469)
I20260808 10:06:54.298658 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.295930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:42995 (local address 127.24.162.129:41469)
0808 10:06:54.295973 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.295975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.296082 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:06:54.296125 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.296126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.296140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.296194 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:54.296413 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.296922 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.297917 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.298274 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.298343 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.298376 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.298541 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.298585 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:06:54.298666 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.295806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.130:37399 (local address 127.24.162.131:56331)
0808 10:06:54.295877 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.295888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.295932 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.296094 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.296095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.296097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:54.296179 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.296181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.296954 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.296955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.297805 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.297809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.298410 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.298411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.298468 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.298471 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.298548 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.298586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":62}
I20260808 10:06:54.298676 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.295821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41469 (local address 127.24.162.131:42995)
0808 10:06:54.295996 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.296006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:06:54.296070 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:06:54.296254 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:06:54.296255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:06:54.296257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:06:54.296335 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.296337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.297026 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.297027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.297810 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:06:54.297812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.298375 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.298377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:06:54.298459 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.298463 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:06:54.298506 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 10:06:54.298570 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260808 10:06:54.298697 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.295896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.162.131:56331 (local address 127.24.162.130:37399)
0808 10:06:54.295943 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.295945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.295955 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:54.295991 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.295991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.296013 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.296078 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:06:54.296264 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.296866 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.297923 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.298289 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.298382 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.298389 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.298570 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.298608 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:06:54.299034 21619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a629c05239654b9fb81c15f399a65199" candidate_uuid: "93bc9fdec7b740f4bc031562ba00ac50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cac21230db4e419016e8867a32b9fe" is_pre_election: true
I20260808 10:06:54.299075 21418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a629c05239654b9fb81c15f399a65199" candidate_uuid: "93bc9fdec7b740f4bc031562ba00ac50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac98d46427a45b5a3e958df45098f20" is_pre_election: true
I20260808 10:06:54.299214 21619 raft_consensus.cc:2454] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93bc9fdec7b740f4bc031562ba00ac50 in term 0.
I20260808 10:06:54.299211 21418 raft_consensus.cc:2454] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93bc9fdec7b740f4bc031562ba00ac50 in term 0.
I20260808 10:06:54.299445 21714 leader_election.cc:302] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93bc9fdec7b740f4bc031562ba00ac50, f1cac21230db4e419016e8867a32b9fe; no voters:
I20260808 10:06:54.299579 21916 raft_consensus.cc:2781] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:06:54.299642 21916 raft_consensus.cc:482] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:06:54.299677 21916 raft_consensus.cc:3037] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:54.300416 21916 raft_consensus.cc:504] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.300572 21916 leader_election.cc:288] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [CANDIDATE]: Term 1 election: Requested vote from peers f1cac21230db4e419016e8867a32b9fe (127.24.162.130:37399), 3ac98d46427a45b5a3e958df45098f20 (127.24.162.129:41469)
I20260808 10:06:54.300755 21619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a629c05239654b9fb81c15f399a65199" candidate_uuid: "93bc9fdec7b740f4bc031562ba00ac50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1cac21230db4e419016e8867a32b9fe"
I20260808 10:06:54.300770 21418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a629c05239654b9fb81c15f399a65199" candidate_uuid: "93bc9fdec7b740f4bc031562ba00ac50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac98d46427a45b5a3e958df45098f20"
I20260808 10:06:54.300850 21619 raft_consensus.cc:3037] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:54.300854 21418 raft_consensus.cc:3037] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:54.301617 21619 raft_consensus.cc:2454] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93bc9fdec7b740f4bc031562ba00ac50 in term 1.
I20260808 10:06:54.301627 21418 raft_consensus.cc:2454] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93bc9fdec7b740f4bc031562ba00ac50 in term 1.
I20260808 10:06:54.301855 21712 leader_election.cc:302] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac98d46427a45b5a3e958df45098f20, 93bc9fdec7b740f4bc031562ba00ac50; no voters:
I20260808 10:06:54.301960 21916 raft_consensus.cc:2781] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:54.302125 21916 raft_consensus.cc:686] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 1 LEADER]: Becoming Leader. State: Replica: 93bc9fdec7b740f4bc031562ba00ac50, State: Running, Role: LEADER
I20260808 10:06:54.302234 21916 consensus_queue.cc:237] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } }
I20260808 10:06:54.303107 21206 catalog_manager.cc:5495] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93bc9fdec7b740f4bc031562ba00ac50 (127.24.162.131). New cstate: current_term: 1 leader_uuid: "93bc9fdec7b740f4bc031562ba00ac50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93bc9fdec7b740f4bc031562ba00ac50" member_type: VOTER last_known_addr { host: "127.24.162.131" port: 46553 } health_report { overall_health: HEALTHY } } }
I20260808 10:06:54.321628 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.319173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.24.162.131:46553)
0808 10:06:54.319233 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:54.319235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:54.319245 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:06:54.319334 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:54.319335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:54.319362 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:54.319443 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:06:54.319577 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.320050 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.320830 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:54.321025 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:54.321106 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:54.321274 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:06:54.321378 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:06:54.321476 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:54.321495 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 10:06:54.321514 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:06:54.323520 21619 raft_consensus.cc:1261] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 1 FOLLOWER]: Refusing update from remote peer 93bc9fdec7b740f4bc031562ba00ac50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:54.323520 21418 raft_consensus.cc:1261] T a629c05239654b9fb81c15f399a65199 P 3ac98d46427a45b5a3e958df45098f20 [term 1 FOLLOWER]: Refusing update from remote peer 93bc9fdec7b740f4bc031562ba00ac50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:06:54.323891 21916 consensus_queue.cc:1035] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ac98d46427a45b5a3e958df45098f20" member_type: VOTER last_known_addr { host: "127.24.162.129" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:54.324036 21923 consensus_queue.cc:1035] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1cac21230db4e419016e8867a32b9fe" member_type: VOTER last_known_addr { host: "127.24.162.130" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:06:54.326596 21933 mvcc.cc:205] Tried to move back new op lower bound from 7316208084266246144 to 7316208084182573056. Current Snapshot: MvccSnapshot[applied={T|T < 7316208084266246144}]
I20260808 10:06:54.326718 21930 mvcc.cc:205] Tried to move back new op lower bound from 7316208084266246144 to 7316208084182573056. Current Snapshot: MvccSnapshot[applied={T|T < 7316208084266246144}]
I20260808 10:06:54.326812 21931 mvcc.cc:205] Tried to move back new op lower bound from 7316208084266246144 to 7316208084182573056. Current Snapshot: MvccSnapshot[applied={T|T < 7316208084266246144}]
I20260808 10:06:54.329124 21207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:54.329224 21207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:06:54.330009 21207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc927c803ce84211b04ddccd5f8bb951: Sending DeleteTablet for 3 replicas of tablet a629c05239654b9fb81c15f399a65199
I20260808 10:06:54.330349 21590 tablet_service.cc:1514] Processing DeleteTablet for tablet a629c05239654b9fb81c15f399a65199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:54 UTC) from {username='slave'} at 127.0.0.1:59546
I20260808 10:06:54.330497 21793 tablet_service.cc:1514] Processing DeleteTablet for tablet a629c05239654b9fb81c15f399a65199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:54 UTC) from {username='slave'} at 127.0.0.1:34234
I20260808 10:06:54.330591 21380 tablet_service.cc:1514] Processing DeleteTablet for tablet a629c05239654b9fb81c15f399a65199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:06:54 UTC) from {username='slave'} at 127.0.0.1:32822
I20260808 10:06:54.330628 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 21291
I20260808 10:06:54.335419 21942 tablet_replica.cc:330] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: stopping tablet replica
I20260808 10:06:54.335589 21942 raft_consensus.cc:2229] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:06:54.335759 21942 raft_consensus.cc:2258] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:54.336787 21942 ts_tablet_manager.cc:1905] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:54.339016 21942 ts_tablet_manager.cc:1918] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:54.339174 21942 log.cc:1196] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-1/wal/wals/a629c05239654b9fb81c15f399a65199
I20260808 10:06:54.339581 21942 ts_tablet_manager.cc:1939] T a629c05239654b9fb81c15f399a65199 P f1cac21230db4e419016e8867a32b9fe: Deleting consensus metadata
I20260808 10:06:54.340402 21192 catalog_manager.cc:4841] TS f1cac21230db4e419016e8867a32b9fe (127.24.162.130:37399): tablet a629c05239654b9fb81c15f399a65199 (table test-table [id=2e74d57174d842b88f5d9750a71015b5]) successfully deleted
I20260808 10:06:54.342679 21943 tablet_replica.cc:330] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: stopping tablet replica
I20260808 10:06:54.342803 21943 raft_consensus.cc:2229] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:06:54.342978 21943 raft_consensus.cc:2258] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:06:54.344300 21943 ts_tablet_manager.cc:1905] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:06:54.345980 21943 ts_tablet_manager.cc:1918] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:06:54.346045 21943 log.cc:1196] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Deleting WAL directory at /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183366714887-25226-0/minicluster-data/ts-2/wal/wals/a629c05239654b9fb81c15f399a65199
I20260808 10:06:54.346308 21943 ts_tablet_manager.cc:1939] T a629c05239654b9fb81c15f399a65199 P 93bc9fdec7b740f4bc031562ba00ac50: Deleting consensus metadata
I20260808 10:06:54.346796 21190 catalog_manager.cc:4841] TS 93bc9fdec7b740f4bc031562ba00ac50 (127.24.162.131:46553): tablet a629c05239654b9fb81c15f399a65199 (table test-table [id=2e74d57174d842b88f5d9750a71015b5]) successfully deleted
W20260808 10:06:54.348021 21190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.162.129:41469 (error 108)
I20260808 10:06:54.348572 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:54.348412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.129:41469 (local address 127.0.0.1:32834)
0808 10:06:54.348483 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:06:54.348530 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:06:54.348667 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 21495
W20260808 10:06:54.351711 21190 catalog_manager.cc:4584] TS 3ac98d46427a45b5a3e958df45098f20 (127.24.162.129:41469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a629c05239654b9fb81c15f399a65199: Network error: Client connection negotiation failed: client connection to 127.24.162.129:41469: connect: Connection refused (error 111)
I20260808 10:06:54.357059 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 21688
I20260808 10:06:54.370884 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 21162
2026-08-08T10:06:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (777 ms)
[----------] 6 tests from EncryptionPolicyTest (9461 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:54 dist-test-slave-nfb5 krb5kdc[21948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:54 dist-test-slave-nfb5 krb5kdc[21948](info): set up 2 sockets
Aug 08 10:06:54 dist-test-slave-nfb5 krb5kdc[21948](info): commencing operation
krb5kdc: starting...
W20260808 10:06:55.606745 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:06:55.734447 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:55.818115 21973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:55.818322 21973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:55.818347 21973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:55.819599 21973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:55.819644 21973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:55.819658 21973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:55.819672 21973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:55.819708 21973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:55.821235 21973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:55.821509 21973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:55.821753 21973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:55.826737 21978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:55.826756 21980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:55.826773 21982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:55.826949 21973 server_base.cc:1034] running on GCE node
I20260808 10:06:55.827266 21973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:55.827564 21973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:55.828740 21973 hybrid_clock.cc:648] HybridClock initialized: now 1786183615828739 us; error 61 us; skew 500 ppm
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183615, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:55.831382 21973 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:06:55.831799 21973 webserver.cc:460] Webserver started at http://127.0.0.1:43751/ using document root <none> and password file <none>
I20260808 10:06:55.832031 21973 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:55.832103 21973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:55.832255 21973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:55.833312 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "11a890344b784de1bae78d3a09e98c8c"
format_stamp: "Formatted at 2026-08-08 10:06:55 on dist-test-slave-nfb5"
server_key: "ab8f19ee5523b63da2104a2ebe860c26"
server_key_iv: "d59c75b17883d4fc205c80213ee7ff63"
server_key_version: "encryptionkey@0"
I20260808 10:06:55.833683 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "11a890344b784de1bae78d3a09e98c8c"
format_stamp: "Formatted at 2026-08-08 10:06:55 on dist-test-slave-nfb5"
server_key: "ab8f19ee5523b63da2104a2ebe860c26"
server_key_iv: "d59c75b17883d4fc205c80213ee7ff63"
server_key_version: "encryptionkey@0"
I20260808 10:06:55.835085 21973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:06:55.835814 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:55.835983 21973 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:06:55.836086 21973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "11a890344b784de1bae78d3a09e98c8c"
format_stamp: "Formatted at 2026-08-08 10:06:55 on dist-test-slave-nfb5"
server_key: "ab8f19ee5523b63da2104a2ebe860c26"
server_key_iv: "d59c75b17883d4fc205c80213ee7ff63"
server_key_version: "encryptionkey@0"
I20260808 10:06:55.836171 21973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:55.849797 21973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:55.852173 21973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:55.852352 21973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:55.856637 21973 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34669
I20260808 10:06:55.856673 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34669 every 8 connection(s)
I20260808 10:06:55.857023 21973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:55.857903 22068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:55.860229 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 21973
I20260808 10:06:55.860330 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:55.860407 22068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c: Bootstrap starting.
I20260808 10:06:55.860535 25226 external_mini_cluster.cc:1428] Setting key 81a533c47f099c17883a600494ac260c
I20260808 10:06:55.861168 22068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:55.861505 22068 log.cc:824] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:55.862284 22068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c: No bootstrap required, opened a new log
I20260808 10:06:55.864584 22068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 10:06:55.864750 22068 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:55.864780 22068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a890344b784de1bae78d3a09e98c8c, State: Initialized, Role: FOLLOWER
I20260808 10:06:55.864900 22068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 10:06:55.864982 22068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:55.865031 22068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:55.865101 22068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:55.865737 22068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 10:06:55.865864 22068 leader_election.cc:302] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a890344b784de1bae78d3a09e98c8c; no voters:
I20260808 10:06:55.866050 22068 leader_election.cc:288] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:55.866074 22078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:55.866195 22078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [term 1 LEADER]: Becoming Leader. State: Replica: 11a890344b784de1bae78d3a09e98c8c, State: Running, Role: LEADER
I20260808 10:06:55.866282 22078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 10:06:55.866334 22068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:55.866631 22078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a890344b784de1bae78d3a09e98c8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260808 10:06:55.866708 22078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:55.866863 22078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a890344b784de1bae78d3a09e98c8c. Latest consensus state: current_term: 1 leader_uuid: "11a890344b784de1bae78d3a09e98c8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a890344b784de1bae78d3a09e98c8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260808 10:06:55.866958 22078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:55.867733 22090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:06:55.868290 22090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:55.870043 22090 catalog_manager.cc:1347] Generated new cluster ID: 0620deb9c27e47a09a5990732f33c123
I20260808 10:06:55.870105 22090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:55.872709 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:55.861137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51987 (local address 127.0.0.1:34669)
0808 10:06:55.861359 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:55.861368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:55.861402 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:06:55.862028 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:55.862041 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:55.862091 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:55.862211 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:55.863050 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:55.863911 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:55.865012 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:55.865510 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:55.865595 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:55.868659 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:55.868664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:55.868665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:55.868675 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:55.870713 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:55.871236 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:55.871239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:55.871240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:55.871346 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:55.871612 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:55.871613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:55.871614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:55.871922 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:55.872092 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:55.872388 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:06:55.872497 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260808 10:06:55.873989 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:06:55.882632 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:55.875570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36481 (local address 127.0.0.1:34669)
0808 10:06:55.875604 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:55.875606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:55.875614 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:06:55.875632 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:55.875633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:55.875641 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:55.875707 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:55.876297 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:55.877065 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:55.878035 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:55.878325 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:55.878399 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:55.879758 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:55.879759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:55.879759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:06:55.879767 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:55.881376 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:55.881788 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:55.881789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:55.881790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:55.881856 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:06:55.882096 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:06:55.882096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:06:55.882097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:06:55.882219 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:55.882323 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:55.882471 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:06:55.882537 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:06:55.883388 22090 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:55.883806 22090 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:55.887007 22090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a890344b784de1bae78d3a09e98c8c: Generated new TSK 0
I20260808 10:06:55.887190 22090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:55.889341 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 21973
W20260808 10:06:55.902568 21945 connection.cc:535] client connection to 127.0.0.1:34669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34669): Connection reset by peer (error 104)
2026-08-08T10:06:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[22119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[22119](info): set up 2 sockets
Aug 08 10:06:55 dist-test-slave-nfb5 krb5kdc[22119](info): commencing operation
krb5kdc: starting...
W20260808 10:06:57.112609 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:57 dist-test-slave-nfb5 krb5kdc[22119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:06:57.237174 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:57.320691 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:57.320891 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:57.320914 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:57.322122 22135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:57.322167 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:57.322182 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:57.322196 22135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:57.322208 22135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:57.323750 22135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:57.324002 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:57.324231 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:57.329174 22145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:57.329187 22143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:57.329167 22142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:57.329350 22135 server_base.cc:1034] running on GCE node
I20260808 10:06:57.329576 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:57.329869 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:57.330976 22135 hybrid_clock.cc:648] HybridClock initialized: now 1786183617330955 us; error 36 us; skew 500 ppm
Aug 08 10:06:57 dist-test-slave-nfb5 krb5kdc[22119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:57.333484 22135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:06:57.333945 22135 webserver.cc:460] Webserver started at http://127.0.0.1:43437/ using document root <none> and password file <none>
I20260808 10:06:57.334169 22135 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:57.334214 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:57.334349 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:57.335508 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "9d17fddcf1874cafadb6d87458d9c0ac"
format_stamp: "Formatted at 2026-08-08 10:06:57 on dist-test-slave-nfb5"
server_key: "9fe71d6624e1fad9c1bbc97751fc0ee8"
server_key_iv: "368dd6cb33ec2ec46c9c3795d5548307"
server_key_version: "encryptionkey@0"
I20260808 10:06:57.335875 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "9d17fddcf1874cafadb6d87458d9c0ac"
format_stamp: "Formatted at 2026-08-08 10:06:57 on dist-test-slave-nfb5"
server_key: "9fe71d6624e1fad9c1bbc97751fc0ee8"
server_key_iv: "368dd6cb33ec2ec46c9c3795d5548307"
server_key_version: "encryptionkey@0"
I20260808 10:06:57.337245 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:06:57.338006 22153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:57.338220 22135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:57.338344 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "9d17fddcf1874cafadb6d87458d9c0ac"
format_stamp: "Formatted at 2026-08-08 10:06:57 on dist-test-slave-nfb5"
server_key: "9fe71d6624e1fad9c1bbc97751fc0ee8"
server_key_iv: "368dd6cb33ec2ec46c9c3795d5548307"
server_key_version: "encryptionkey@0"
I20260808 10:06:57.338424 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:57.349217 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:57.351086 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:57.351243 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:57.355544 22135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44709
I20260808 10:06:57.355665 22228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44709 every 8 connection(s)
I20260808 10:06:57.355976 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:57.356859 22231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:57.359387 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac: Bootstrap starting.
I20260808 10:06:57.359993 22231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:57.360293 22231 log.cc:824] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:57.361045 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac: No bootstrap required, opened a new log
I20260808 10:06:57.362360 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22135
I20260808 10:06:57.362478 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:57.362671 25226 external_mini_cluster.cc:1428] Setting key b5cd374c0ecbd0f3eb91e35d7bd624c2
I20260808 10:06:57.362769 22231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260808 10:06:57.362965 22231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:57.363021 22231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d17fddcf1874cafadb6d87458d9c0ac, State: Initialized, Role: FOLLOWER
I20260808 10:06:57.363166 22231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260808 10:06:57.363260 22231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:57.363293 22231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:57.363343 22231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:57.364408 22231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260808 10:06:57.364500 22231 leader_election.cc:302] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d17fddcf1874cafadb6d87458d9c0ac; no voters:
I20260808 10:06:57.364698 22231 leader_election.cc:288] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:57.364740 22238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:57.364886 22238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [term 1 LEADER]: Becoming Leader. State: Replica: 9d17fddcf1874cafadb6d87458d9c0ac, State: Running, Role: LEADER
I20260808 10:06:57.364984 22231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:57.364971 22238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260808 10:06:57.365305 22238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d17fddcf1874cafadb6d87458d9c0ac. Latest consensus state: current_term: 1 leader_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } } }
I20260808 10:06:57.365406 22238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:57.365592 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d17fddcf1874cafadb6d87458d9c0ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } } }
I20260808 10:06:57.365664 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:57.365779 22246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:57.366298 22246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:57.368233 22246 catalog_manager.cc:1347] Generated new cluster ID: 4799cc50927d40c9a0e3607d2ccfcd09
I20260808 10:06:57.368291 22246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:57.368798 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:57.363232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60385 (local address 127.0.0.1:44709)
0808 10:06:57.363439 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:57.363449 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:57.363486 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:06:57.363599 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:57.363604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:57.363653 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:57.363760 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:57.364036 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:57.365201 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:57.366406 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:57.367064 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:57.367188 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:57.368026 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:57.368029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:57.368030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:57.368235 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:57.368276 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:57.368292 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:57.368380 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:57.368545 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:06:57.368577 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":122,"threads_started":1}
I20260808 10:06:57.369904 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:06:57.372859 22246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:57.373323 22246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:57.378101 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:57.371515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46589 (local address 127.0.0.1:44709)
0808 10:06:57.371549 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:57.371551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:57.372549 (+ 998us) server_negotiation.cc:407] Connection header received
0808 10:06:57.372637 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:57.372637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:57.372649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:57.372713 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:57.375021 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:57.375734 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:57.376746 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:06:57.377060 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:06:57.377138 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:06:57.377850 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:57.377851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:57.377852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:57.377856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:57.377872 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:57.377877 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:57.377943 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:57.378015 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 10:06:57.378028 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:06:57.380518 22246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d17fddcf1874cafadb6d87458d9c0ac: Generated new TSK 0
I20260808 10:06:57.380693 22246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:57.384516 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22135
W20260808 10:06:57.398315 22117 connection.cc:535] client connection to 127.0.0.1:44709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44709): Connection reset by peer (error 104)
2026-08-08T10:06:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:57 dist-test-slave-nfb5 krb5kdc[22274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:57 dist-test-slave-nfb5 krb5kdc[22274](info): set up 2 sockets
Aug 08 10:06:57 dist-test-slave-nfb5 krb5kdc[22274](info): commencing operation
krb5kdc: starting...
W20260808 10:06:58.601606 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:06:58 dist-test-slave-nfb5 krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:06:58.726962 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:06:58.813380 22294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:06:58.813639 22294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:06:58.813679 22294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:06:58.814831 22294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:06:58.814903 22294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:06:58.814925 22294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:06:58.814987 22294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:06:58.815016 22294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:06:58.816538 22294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:06:58.816841 22294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:06:58.817078 22294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:06:58.821973 22304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:58.821985 22301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:06:58.821987 22302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:06:58.822289 22294 server_base.cc:1034] running on GCE node
I20260808 10:06:58.822510 22294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:06:58.822745 22294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:06:58.823915 22294 hybrid_clock.cc:648] HybridClock initialized: now 1786183618823896 us; error 31 us; skew 500 ppm
Aug 08 10:06:58 dist-test-slave-nfb5 krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:06:58.826380 22294 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:06:58.826877 22294 webserver.cc:460] Webserver started at http://127.0.0.1:36089/ using document root <none> and password file <none>
I20260808 10:06:58.827152 22294 fs_manager.cc:359] Metadata directory not provided
I20260808 10:06:58.827199 22294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:06:58.827291 22294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:06:58.828303 22294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "2870bb5f54de4d868590105ffd902daa"
format_stamp: "Formatted at 2026-08-08 10:06:58 on dist-test-slave-nfb5"
server_key: "bc663a1f3b3bafe5692f84db1130d159"
server_key_iv: "a1fe3e6247c360493a387baf6d8bacac"
server_key_version: "encryptionkey@0"
I20260808 10:06:58.828608 22294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "2870bb5f54de4d868590105ffd902daa"
format_stamp: "Formatted at 2026-08-08 10:06:58 on dist-test-slave-nfb5"
server_key: "bc663a1f3b3bafe5692f84db1130d159"
server_key_iv: "a1fe3e6247c360493a387baf6d8bacac"
server_key_version: "encryptionkey@0"
I20260808 10:06:58.829860 22294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:06:58.830610 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:06:58.830878 22294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:06:58.830987 22294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "2870bb5f54de4d868590105ffd902daa"
format_stamp: "Formatted at 2026-08-08 10:06:58 on dist-test-slave-nfb5"
server_key: "bc663a1f3b3bafe5692f84db1130d159"
server_key_iv: "a1fe3e6247c360493a387baf6d8bacac"
server_key_version: "encryptionkey@0"
I20260808 10:06:58.831046 22294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:06:58.834194 22294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:06:58.836470 22294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:06:58.836580 22294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:06:58.840556 22294 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43265
I20260808 10:06:58.840662 22388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43265 every 8 connection(s)
I20260808 10:06:58.840955 22294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:06:58.841984 22389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:06:58.842397 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22294
I20260808 10:06:58.842494 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:06:58.842705 25226 external_mini_cluster.cc:1428] Setting key 964c1035111185cf4305aef13b1afb73
I20260808 10:06:58.845060 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:58.843272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57605 (local address 127.0.0.1:43265)
0808 10:06:58.843457 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:58.843466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:58.843496 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:06:58.843895 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:58.843901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:58.843926 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:58.844043 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:58.844679 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:58.844683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:58.844683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:58.844688 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:58.844736 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:58.844746 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:58.844804 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:58.844905 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:06:58.844938 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260808 10:06:58.845121 22389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa: Bootstrap starting.
I20260808 10:06:58.845896 22389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa: Neither blocks nor log segments found. Creating new log.
I20260808 10:06:58.846087 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:06:58.846237 22389 log.cc:824] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:06:58.847012 22389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa: No bootstrap required, opened a new log
I20260808 10:06:58.848563 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:06:58.847633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45071 (local address 127.0.0.1:43265)
0808 10:06:58.847664 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 10:06:58.847666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:06:58.847761 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:06:58.847830 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:06:58.847830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:06:58.847833 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:06:58.847882 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:06:58.848396 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:06:58.848397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:06:58.848397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:06:58.848401 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:06:58.848412 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:06:58.848415 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:06:58.848460 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:06:58.848513 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 10:06:58.848523 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 10:06:58.849611 22389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } }
I20260808 10:06:58.849743 22389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:06:58.849766 22389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2870bb5f54de4d868590105ffd902daa, State: Initialized, Role: FOLLOWER
I20260808 10:06:58.849918 22389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } }
I20260808 10:06:58.850004 22389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:06:58.850052 22389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:06:58.850103 22389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:06:58.850740 22389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } }
I20260808 10:06:58.850870 22389 leader_election.cc:302] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2870bb5f54de4d868590105ffd902daa; no voters:
I20260808 10:06:58.851058 22389 leader_election.cc:288] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:06:58.851106 22405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:06:58.851277 22405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [term 1 LEADER]: Becoming Leader. State: Replica: 2870bb5f54de4d868590105ffd902daa, State: Running, Role: LEADER
I20260808 10:06:58.851329 22389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:06:58.851394 22405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } }
I20260808 10:06:58.851770 22405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2870bb5f54de4d868590105ffd902daa. Latest consensus state: current_term: 1 leader_uuid: "2870bb5f54de4d868590105ffd902daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } } }
I20260808 10:06:58.851780 22408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2870bb5f54de4d868590105ffd902daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2870bb5f54de4d868590105ffd902daa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43265 } } }
I20260808 10:06:58.851877 22405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:58.851886 22408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa [sys.catalog]: This master's current role is: LEADER
I20260808 10:06:58.852082 22417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:06:58.852648 22417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:06:58.855947 22417 catalog_manager.cc:1347] Generated new cluster ID: eaf5f40fb07e41069b3ab978c31fcccd
I20260808 10:06:58.856001 22417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:06:58.869557 22417 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:06:58.869743 22417 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:06:58.876727 22417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2870bb5f54de4d868590105ffd902daa: Generated new TSK 0
I20260808 10:06:58.876929 22417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:06:58.882352 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22294
2026-08-08T10:06:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:06:58 dist-test-slave-nfb5 krb5kdc[22434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:06:58 dist-test-slave-nfb5 krb5kdc[22434](info): set up 2 sockets
Aug 08 10:06:58 dist-test-slave-nfb5 krb5kdc[22434](info): commencing operation
krb5kdc: starting...
W20260808 10:07:00.099071 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:00.221617 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:00.304154 22450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:00.304364 22450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:00.304388 22450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:00.305642 22450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:00.305684 22450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:00.305699 22450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:00.305712 22450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:00.305724 22450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:00.307272 22450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:00.307483 22450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:00.307662 22450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:00.312538 22458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:00.312548 22459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:00.312547 22462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:00.312736 22450 server_base.cc:1034] running on GCE node
I20260808 10:07:00.313024 22450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:00.313280 22450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:00.314422 22450 hybrid_clock.cc:648] HybridClock initialized: now 1786183620314405 us; error 34 us; skew 500 ppm
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:00.323771 22450 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:00.324333 22450 webserver.cc:460] Webserver started at http://127.0.0.1:36029/ using document root <none> and password file <none>
I20260808 10:07:00.324599 22450 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:00.324687 22450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:00.324847 22450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:00.326102 22450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "d476c87801044fc68cc7071733738505"
format_stamp: "Formatted at 2026-08-08 10:07:00 on dist-test-slave-nfb5"
server_key: "507901577e3dcc4b56168443b67f7d5c"
server_key_iv: "7e5719828e2285c5455fb7f01ba56b5e"
server_key_version: "encryptionkey@0"
I20260808 10:07:00.326514 22450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "d476c87801044fc68cc7071733738505"
format_stamp: "Formatted at 2026-08-08 10:07:00 on dist-test-slave-nfb5"
server_key: "507901577e3dcc4b56168443b67f7d5c"
server_key_iv: "7e5719828e2285c5455fb7f01ba56b5e"
server_key_version: "encryptionkey@0"
I20260808 10:07:00.328104 22450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 10:07:00.328965 22470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:00.329236 22450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:00.329375 22450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "d476c87801044fc68cc7071733738505"
format_stamp: "Formatted at 2026-08-08 10:07:00 on dist-test-slave-nfb5"
server_key: "507901577e3dcc4b56168443b67f7d5c"
server_key_iv: "7e5719828e2285c5455fb7f01ba56b5e"
server_key_version: "encryptionkey@0"
I20260808 10:07:00.329464 22450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:00.355933 22450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:00.358063 22450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:00.358264 22450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:00.362797 22450 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42955
I20260808 10:07:00.362851 22540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42955 every 8 connection(s)
I20260808 10:07:00.363265 22450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:00.364204 22542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:00.366489 22542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505: Bootstrap starting.
I20260808 10:07:00.367120 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22450
I20260808 10:07:00.367143 22542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:00.367251 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:00.367447 25226 external_mini_cluster.cc:1428] Setting key 7a532b7d5417e6617c3cae699c555776
I20260808 10:07:00.367492 22542 log.cc:824] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:00.368361 22542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505: No bootstrap required, opened a new log
I20260808 10:07:00.370939 22542 raft_consensus.cc:348] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } }
I20260808 10:07:00.371104 22542 raft_consensus.cc:374] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:00.371205 22542 raft_consensus.cc:729] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d476c87801044fc68cc7071733738505, State: Initialized, Role: FOLLOWER
I20260808 10:07:00.371363 22542 consensus_queue.cc:260] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } }
I20260808 10:07:00.371439 22542 raft_consensus.cc:388] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:00.371495 22542 raft_consensus.cc:482] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:00.371542 22542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:00.372129 22542 raft_consensus.cc:504] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } }
I20260808 10:07:00.372231 22542 leader_election.cc:302] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d476c87801044fc68cc7071733738505; no voters:
I20260808 10:07:00.372385 22542 leader_election.cc:288] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:00.372457 22548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:00.372615 22542 sys_catalog.cc:564] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:00.372663 22548 raft_consensus.cc:686] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [term 1 LEADER]: Becoming Leader. State: Replica: d476c87801044fc68cc7071733738505, State: Running, Role: LEADER
I20260808 10:07:00.372838 22548 consensus_queue.cc:237] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } }
I20260808 10:07:00.373256 22549 sys_catalog.cc:455] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d476c87801044fc68cc7071733738505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } } }
I20260808 10:07:00.373338 22549 sys_catalog.cc:458] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:00.373661 22557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:00.374333 22557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:00.373310 22550 sys_catalog.cc:455] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d476c87801044fc68cc7071733738505. Latest consensus state: current_term: 1 leader_uuid: "d476c87801044fc68cc7071733738505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d476c87801044fc68cc7071733738505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42955 } } }
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:07:00.375320 22550 sys_catalog.cc:458] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:00.376463 22557 catalog_manager.cc:1347] Generated new cluster ID: 3975841aea1e45069b875029c5e4ed10
I20260808 10:07:00.376520 22557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:00.379650 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:00.367956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33247 (local address 127.0.0.1:42955)
0808 10:07:00.368129 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:00.368138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:00.368175 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:07:00.368761 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:00.368770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:00.368831 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:00.368980 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:00.369584 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:00.371058 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:00.372019 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:00.372651 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:00.373091 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:00.375841 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:00.375845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:00.375845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:00.375857 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:00.377933 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:00.378345 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:00.378348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:00.378348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:00.378454 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:00.378662 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:00.378663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:00.378664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:00.378996 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:00.379161 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:00.379281 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:07:00.379429 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 10:07:00.380589 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:00.389465 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:00.382222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39477 (local address 127.0.0.1:42955)
0808 10:07:00.382272 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:00.382274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:00.382450 (+ 176us) server_negotiation.cc:407] Connection header received
0808 10:07:00.382507 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:00.382508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:00.382517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:00.382564 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:00.383229 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:00.383961 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:00.384982 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:00.385292 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:00.385351 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:00.386725 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:00.386727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:00.386727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:00.386733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:00.388303 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:00.388708 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:00.388709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:00.388709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:00.388764 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:00.388976 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:00.388976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:00.388977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:00.389086 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:00.389171 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:00.389314 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 10:07:00.389382 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:00.399894 22557 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:00.400609 22557 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:00.411405 22557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d476c87801044fc68cc7071733738505: Generated new TSK 0
I20260808 10:07:00.411602 22557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:00.417318 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22450
2026-08-08T10:07:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22578](info): set up 2 sockets
Aug 08 10:07:00 dist-test-slave-nfb5 krb5kdc[22578](info): commencing operation
krb5kdc: starting...
W20260808 10:07:01.641340 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:01 dist-test-slave-nfb5 krb5kdc[22578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:01.768360 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:01.851136 22593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:01.851366 22593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:01.851404 22593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:01.852528 22593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:01.852600 22593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:01.852624 22593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:01.852646 22593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:01.852681 22593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:01.854172 22593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:01.854460 22593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:01.854699 22593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:01.859656 22606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:01.859638 22602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:01.859627 22601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:01.859717 22593 server_base.cc:1034] running on GCE node
I20260808 10:07:01.860106 22593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:01.860371 22593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:01.861565 22593 hybrid_clock.cc:648] HybridClock initialized: now 1786183621861557 us; error 38 us; skew 500 ppm
Aug 08 10:07:01 dist-test-slave-nfb5 krb5kdc[22578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:01.864614 22593 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:01.865139 22593 webserver.cc:460] Webserver started at http://127.0.0.1:46321/ using document root <none> and password file <none>
I20260808 10:07:01.865384 22593 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:01.865456 22593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:01.865593 22593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:01.866703 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "d28f223cb4f74c1888d2a6c4833ee829"
format_stamp: "Formatted at 2026-08-08 10:07:01 on dist-test-slave-nfb5"
server_key: "d79b8df6323e724fc2837ea38ee083d2"
server_key_iv: "88fcdbdd1ea747b45e28dd21e6b578c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:01.867118 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "d28f223cb4f74c1888d2a6c4833ee829"
format_stamp: "Formatted at 2026-08-08 10:07:01 on dist-test-slave-nfb5"
server_key: "d79b8df6323e724fc2837ea38ee083d2"
server_key_iv: "88fcdbdd1ea747b45e28dd21e6b578c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:01.868522 22593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:01.869259 22618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:01.869540 22593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:01.869647 22593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "d28f223cb4f74c1888d2a6c4833ee829"
format_stamp: "Formatted at 2026-08-08 10:07:01 on dist-test-slave-nfb5"
server_key: "d79b8df6323e724fc2837ea38ee083d2"
server_key_iv: "88fcdbdd1ea747b45e28dd21e6b578c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:01.869726 22593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:01.883014 22593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:01.884886 22593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:01.885057 22593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:01.889345 22593 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37749
I20260808 10:07:01.889380 22696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37749 every 8 connection(s)
I20260808 10:07:01.889753 22593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:01.890626 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:01.892946 22697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829: Bootstrap starting.
I20260808 10:07:01.893595 22697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:01.893821 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22593
I20260808 10:07:01.893942 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:01.893999 22697 log.cc:824] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:01.894145 25226 external_mini_cluster.cc:1428] Setting key fdb1a7dc18145865e8a95489a4caa9f8
I20260808 10:07:01.894891 22697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829: No bootstrap required, opened a new log
I20260808 10:07:01.897066 22697 raft_consensus.cc:348] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260808 10:07:01.897228 22697 raft_consensus.cc:374] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:01.897277 22697 raft_consensus.cc:729] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d28f223cb4f74c1888d2a6c4833ee829, State: Initialized, Role: FOLLOWER
I20260808 10:07:01.897428 22697 consensus_queue.cc:260] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260808 10:07:01.897547 22697 raft_consensus.cc:388] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:01.897596 22697 raft_consensus.cc:482] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:01.897639 22697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:01.898227 22697 raft_consensus.cc:504] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260808 10:07:01.898351 22697 leader_election.cc:302] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d28f223cb4f74c1888d2a6c4833ee829; no voters:
I20260808 10:07:01.898524 22697 leader_election.cc:288] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:01.898557 22704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:01.898754 22704 raft_consensus.cc:686] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [term 1 LEADER]: Becoming Leader. State: Replica: d28f223cb4f74c1888d2a6c4833ee829, State: Running, Role: LEADER
I20260808 10:07:01.898792 22697 sys_catalog.cc:564] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:01.898906 22704 consensus_queue.cc:237] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260808 10:07:01.899353 22706 sys_catalog.cc:455] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d28f223cb4f74c1888d2a6c4833ee829. Latest consensus state: current_term: 1 leader_uuid: "d28f223cb4f74c1888d2a6c4833ee829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } } }
I20260808 10:07:01.899371 22705 sys_catalog.cc:455] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d28f223cb4f74c1888d2a6c4833ee829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d28f223cb4f74c1888d2a6c4833ee829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } } }
I20260808 10:07:01.899564 22706 sys_catalog.cc:458] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:01.899817 22705 sys_catalog.cc:458] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:01.899801 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:01.894647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57877 (local address 127.0.0.1:37749)
0808 10:07:01.894844 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:01.894854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:01.894890 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:07:01.895392 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:01.895399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:01.895446 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:01.895598 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:01.895792 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:01.896670 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:01.897551 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:01.898056 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:01.898188 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:01.898879 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:01.898883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:01.898883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:01.899115 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:01.899156 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:01.899169 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:01.899265 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:01.899531 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 10:07:01.899600 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260808 10:07:01.901180 22715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:01.901219 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:01.901700 22715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:01.903427 22715 catalog_manager.cc:1347] Generated new cluster ID: 23d3cbfaef7b492f86aacaa9c036a0fe
I20260808 10:07:01.903486 22715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:01.906266 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:01.902684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34451 (local address 127.0.0.1:37749)
0808 10:07:01.902720 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:01.902722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:01.902941 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:07:01.903006 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:01.903007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:01.903016 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:01.903095 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:01.903246 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:01.903926 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:01.904923 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:01.905211 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:01.905260 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:01.905984 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:01.905985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:01.905986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:01.905989 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:01.906002 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:01.906006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:01.906042 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:01.906181 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 10:07:01.906198 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:01.924391 22715 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:01.924981 22715 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:01.937628 22715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d28f223cb4f74c1888d2a6c4833ee829: Generated new TSK 0
I20260808 10:07:01.937894 22715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:01.944875 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22593
2026-08-08T10:07:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:02 dist-test-slave-nfb5 krb5kdc[22744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:02 dist-test-slave-nfb5 krb5kdc[22744](info): set up 2 sockets
Aug 08 10:07:02 dist-test-slave-nfb5 krb5kdc[22744](info): commencing operation
krb5kdc: starting...
W20260808 10:07:03.169523 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:03 dist-test-slave-nfb5 krb5kdc[22744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:03.301378 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:03.386550 22758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:03.386760 22758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:03.386785 22758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:03.387925 22758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:03.387969 22758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:03.387984 22758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:03.387996 22758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:03.388008 22758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:03.389627 22758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:03.389873 22758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:03.390044 22758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:03.394950 22771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:03.394979 22767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:03.394948 22768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:03.395665 22758 server_base.cc:1034] running on GCE node
I20260808 10:07:03.395934 22758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:03.396286 22758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:03.397436 22758 hybrid_clock.cc:648] HybridClock initialized: now 1786183623397411 us; error 40 us; skew 500 ppm
Aug 08 10:07:03 dist-test-slave-nfb5 krb5kdc[22744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:03.400130 22758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:03.400629 22758 webserver.cc:460] Webserver started at http://127.0.0.1:39813/ using document root <none> and password file <none>
I20260808 10:07:03.400897 22758 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:03.400969 22758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:03.401125 22758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:03.402199 22758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "4cd11db2b98e416998c538f71aab0d61"
format_stamp: "Formatted at 2026-08-08 10:07:03 on dist-test-slave-nfb5"
server_key: "083d7675149a9f135d8480e40997c636"
server_key_iv: "8243c8926b339d9f5e1f9141245408be"
server_key_version: "encryptionkey@0"
I20260808 10:07:03.402573 22758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "4cd11db2b98e416998c538f71aab0d61"
format_stamp: "Formatted at 2026-08-08 10:07:03 on dist-test-slave-nfb5"
server_key: "083d7675149a9f135d8480e40997c636"
server_key_iv: "8243c8926b339d9f5e1f9141245408be"
server_key_version: "encryptionkey@0"
I20260808 10:07:03.404014 22758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:03.404779 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:03.404982 22758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:03.405093 22758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "4cd11db2b98e416998c538f71aab0d61"
format_stamp: "Formatted at 2026-08-08 10:07:03 on dist-test-slave-nfb5"
server_key: "083d7675149a9f135d8480e40997c636"
server_key_iv: "8243c8926b339d9f5e1f9141245408be"
server_key_version: "encryptionkey@0"
I20260808 10:07:03.405170 22758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:03.408583 22758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:03.410678 22758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:03.410849 22758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:03.415191 22758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36569
I20260808 10:07:03.415238 22852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36569 every 8 connection(s)
I20260808 10:07:03.415576 22758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:03.416606 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22758
I20260808 10:07:03.416635 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:03.416738 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:03.416935 25226 external_mini_cluster.cc:1428] Setting key 22175c5f3eb0b53977aeaace23bdec1c
I20260808 10:07:03.419546 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:03.417494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47515 (local address 127.0.0.1:36569)
0808 10:07:03.417732 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:03.417742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:03.417775 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:07:03.418288 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:03.418297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:03.418332 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:03.418489 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:03.419119 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:03.419122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:03.419123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:03.419127 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:03.419171 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:03.419184 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:03.419262 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:03.419316 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:07:03.419356 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260808 10:07:03.420295 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: Bootstrap starting.
I20260808 10:07:03.420599 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:03.421061 22853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:03.421363 22853 log.cc:824] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:03.422212 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: No bootstrap required, opened a new log
I20260808 10:07:03.423190 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:03.422160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33819 (local address 127.0.0.1:36569)
0808 10:07:03.422205 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:03.422207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:03.422306 (+ 99us) server_negotiation.cc:407] Connection header received
0808 10:07:03.422411 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:03.422412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:03.422415 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:03.422461 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:03.423004 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:03.423005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:03.423005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:03.423009 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:03.423023 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:03.423027 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:03.423083 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:03.423138 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:07:03.423148 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:07:03.424615 22853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } }
I20260808 10:07:03.424749 22853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:03.424772 22853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd11db2b98e416998c538f71aab0d61, State: Initialized, Role: FOLLOWER
I20260808 10:07:03.424916 22853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } }
I20260808 10:07:03.425004 22853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:03.425047 22853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:03.425100 22853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:03.425663 22853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } }
I20260808 10:07:03.425787 22853 leader_election.cc:302] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cd11db2b98e416998c538f71aab0d61; no voters:
I20260808 10:07:03.425961 22853 leader_election.cc:288] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:03.426013 22867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:03.426208 22867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd11db2b98e416998c538f71aab0d61, State: Running, Role: LEADER
I20260808 10:07:03.426232 22853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:03.426357 22867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } }
I20260808 10:07:03.427592 22869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cd11db2b98e416998c538f71aab0d61. Latest consensus state: current_term: 1 leader_uuid: "4cd11db2b98e416998c538f71aab0d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } } }
I20260808 10:07:03.427685 22869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:03.427888 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cd11db2b98e416998c538f71aab0d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd11db2b98e416998c538f71aab0d61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36569 } } }
I20260808 10:07:03.428041 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61 [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:03.428232 22889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:03.428272 22889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:03.428357 22890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:03.428467 22890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:03.429880 22890 catalog_manager.cc:1347] Generated new cluster ID: af786dad86cd49dc9e89d16add640ac7
I20260808 10:07:03.429944 22890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:03.448669 22890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:03.448861 22890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:03.455544 22890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cd11db2b98e416998c538f71aab0d61: Generated new TSK 0
I20260808 10:07:03.455742 22890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:03.458037 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22758
2026-08-08T10:07:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:03 dist-test-slave-nfb5 krb5kdc[22896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:03 dist-test-slave-nfb5 krb5kdc[22896](info): set up 2 sockets
Aug 08 10:07:03 dist-test-slave-nfb5 krb5kdc[22896](info): commencing operation
krb5kdc: starting...
W20260808 10:07:04.959462 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.477s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:05.150307 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:05.282369 22917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:05.282603 22917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:05.282642 22917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:05.283926 22917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:05.283996 22917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:05.284021 22917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:05.284045 22917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:05.284072 22917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:05.285562 22917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:05.285853 22917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:05.286087 22917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:05.291136 22925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:05.291132 22929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:05.291136 22924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:05.291469 22917 server_base.cc:1034] running on GCE node
I20260808 10:07:05.291712 22917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:05.292013 22917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:05.293196 22917 hybrid_clock.cc:648] HybridClock initialized: now 1786183625293187 us; error 38 us; skew 500 ppm
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:05.295853 22917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:05.296298 22917 webserver.cc:460] Webserver started at http://127.0.0.1:40055/ using document root <none> and password file <none>
I20260808 10:07:05.296556 22917 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:05.296629 22917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:05.296789 22917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:05.297847 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "ef391eba825443a28851649719c431e9"
format_stamp: "Formatted at 2026-08-08 10:07:05 on dist-test-slave-nfb5"
server_key: "81c96825689c3a7761a5a8922a226ccf"
server_key_iv: "a473b66e0b8c16a8a37561917210f447"
server_key_version: "encryptionkey@0"
I20260808 10:07:05.298224 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "ef391eba825443a28851649719c431e9"
format_stamp: "Formatted at 2026-08-08 10:07:05 on dist-test-slave-nfb5"
server_key: "81c96825689c3a7761a5a8922a226ccf"
server_key_iv: "a473b66e0b8c16a8a37561917210f447"
server_key_version: "encryptionkey@0"
I20260808 10:07:05.299784 22917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:05.300556 22939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:05.300781 22917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:05.300889 22917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "ef391eba825443a28851649719c431e9"
format_stamp: "Formatted at 2026-08-08 10:07:05 on dist-test-slave-nfb5"
server_key: "81c96825689c3a7761a5a8922a226ccf"
server_key_iv: "a473b66e0b8c16a8a37561917210f447"
server_key_version: "encryptionkey@0"
I20260808 10:07:05.300972 22917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:05.356756 22917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:05.360147 22917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:05.360345 22917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:05.366588 22917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44695
I20260808 10:07:05.366619 23009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44695 every 8 connection(s)
I20260808 10:07:05.367059 22917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:05.368011 23010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:05.369544 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 22917
I20260808 10:07:05.369673 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:05.369844 25226 external_mini_cluster.cc:1428] Setting key abe3420f42b6105d4b8f82b8000846e5
I20260808 10:07:05.370712 23010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9: Bootstrap starting.
I20260808 10:07:05.371381 23010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:05.371690 23010 log.cc:824] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:05.372397 23010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9: No bootstrap required, opened a new log
I20260808 10:07:05.374212 23010 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } }
I20260808 10:07:05.374372 23010 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:05.374418 23010 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef391eba825443a28851649719c431e9, State: Initialized, Role: FOLLOWER
I20260808 10:07:05.374571 23010 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } }
I20260808 10:07:05.374665 23010 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:05.374725 23010 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:05.374780 23010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:05.375396 23010 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } }
I20260808 10:07:05.375521 23010 leader_election.cc:302] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef391eba825443a28851649719c431e9; no voters:
I20260808 10:07:05.375679 23010 leader_election.cc:288] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:05.375701 23017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:05.375891 23017 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [term 1 LEADER]: Becoming Leader. State: Replica: ef391eba825443a28851649719c431e9, State: Running, Role: LEADER
I20260808 10:07:05.376003 23010 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:05.376030 23017 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } }
I20260808 10:07:05.376441 23017 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef391eba825443a28851649719c431e9. Latest consensus state: current_term: 1 leader_uuid: "ef391eba825443a28851649719c431e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } } }
I20260808 10:07:05.376694 23017 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [sys.catalog]: This master's current role is: LEADER
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:07:05.376614 23019 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef391eba825443a28851649719c431e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef391eba825443a28851649719c431e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44695 } } }
I20260808 10:07:05.377054 23019 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:05.377218 23028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:05.377750 23028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:05.380484 23028 catalog_manager.cc:1347] Generated new cluster ID: 67bbbd6b1b27445793935a4ba93e498e
I20260808 10:07:05.380538 23028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:05.382107 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:05.370430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:44807 (local address 127.0.0.1:44695)
0808 10:07:05.370633 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:05.370650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:05.370682 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:07:05.371276 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:05.371284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:05.371335 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:05.371491 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:05.372244 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:05.373225 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:05.374000 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:05.374442 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:05.374550 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:05.377606 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:05.377611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:05.377611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:05.377624 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:05.379839 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:05.380306 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:05.380310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:05.380310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:05.380423 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:05.380720 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:05.380721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:05.380722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:05.381029 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:05.381203 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:05.381759 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 10:07:05.381870 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260808 10:07:05.383041 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:05.391520 23028 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:05.392266 23028 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:05.394531 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:05.384585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:60765 (local address 127.0.0.1:44695)
0808 10:07:05.384619 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:05.384621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:05.387150 (+ 2529us) server_negotiation.cc:407] Connection header received
0808 10:07:05.387215 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:05.387216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:05.387226 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:05.387274 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:05.387906 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:05.388651 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:05.389695 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:05.390012 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:05.390076 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:05.391625 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:05.391627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:05.391627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:05.391634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:05.393252 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:05.393701 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:05.393702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:05.393703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:05.393761 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:05.394001 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:05.394001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:05.394002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:05.394116 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:05.394209 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:05.394382 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 10:07:05.394444 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:07:05.398031 23028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef391eba825443a28851649719c431e9: Generated new TSK 0
I20260808 10:07:05.398234 23028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:05.401546 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 22917
W20260808 10:07:05.418071 22893 connection.cc:535] client connection to 127.0.0.1:44695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44695): Connection reset by peer (error 104)
2026-08-08T10:07:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1949 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[23053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[23053](info): set up 2 sockets
Aug 08 10:07:05 dist-test-slave-nfb5 krb5kdc[23053](info): commencing operation
krb5kdc: starting...
W20260808 10:07:06.627521 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:06 dist-test-slave-nfb5 krb5kdc[23053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:06.753321 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:06.839745 23074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:06.839977 23074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:06.840014 23074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:06.841140 23074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:06.841209 23074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:06.841235 23074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:06.841264 23074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:06.841295 23074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:06.842746 23074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:06.843036 23074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:06.843261 23074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:06.848186 23074 server_base.cc:1034] running on GCE node
W20260808 10:07:06.848109 23085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:06.848114 23080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:06.848134 23079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:06.848654 23074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:06.849157 23074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:06.850359 23074 hybrid_clock.cc:648] HybridClock initialized: now 1786183626850340 us; error 50 us; skew 500 ppm
Aug 08 10:07:06 dist-test-slave-nfb5 krb5kdc[23053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:06.853061 23074 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:06.853497 23074 webserver.cc:460] Webserver started at http://127.0.0.1:33521/ using document root <none> and password file <none>
I20260808 10:07:06.853781 23074 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:06.853859 23074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:06.854008 23074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:06.855132 23074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "ec1529509bad4f8d964523a97195bc73"
format_stamp: "Formatted at 2026-08-08 10:07:06 on dist-test-slave-nfb5"
server_key: "4eaa138e5ae9ba2ac257641609cd2be3"
server_key_iv: "adf7923753b672253394e453f733645e"
server_key_version: "encryptionkey@0"
I20260808 10:07:06.855502 23074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "ec1529509bad4f8d964523a97195bc73"
format_stamp: "Formatted at 2026-08-08 10:07:06 on dist-test-slave-nfb5"
server_key: "4eaa138e5ae9ba2ac257641609cd2be3"
server_key_iv: "adf7923753b672253394e453f733645e"
server_key_version: "encryptionkey@0"
I20260808 10:07:06.856881 23074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:06.857733 23094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:06.857937 23074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:06.858032 23074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "ec1529509bad4f8d964523a97195bc73"
format_stamp: "Formatted at 2026-08-08 10:07:06 on dist-test-slave-nfb5"
server_key: "4eaa138e5ae9ba2ac257641609cd2be3"
server_key_iv: "adf7923753b672253394e453f733645e"
server_key_version: "encryptionkey@0"
I20260808 10:07:06.858134 23074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:06.871270 23074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:06.873158 23074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:06.873332 23074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:06.877526 23074 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33871
I20260808 10:07:06.877565 23163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33871 every 8 connection(s)
I20260808 10:07:06.877914 23074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:06.878551 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23074
I20260808 10:07:06.878683 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:06.878968 25226 external_mini_cluster.cc:1428] Setting key 648039a470c39000e87d4e3c23e701c9
I20260808 10:07:06.879045 23165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:06.882613 23165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: Bootstrap starting.
I20260808 10:07:06.883374 23165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:06.883718 23165 log.cc:824] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:06.884544 23165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: No bootstrap required, opened a new log
I20260808 10:07:06.885413 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:06.879492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:59791 (local address 127.0.0.1:33871)
0808 10:07:06.879726 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:06.879735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:06.879771 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:07:06.880297 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:06.880305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:06.880377 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:06.880532 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:06.880779 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:06.882159 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:06.883058 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:06.883727 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:06.883856 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:06.884656 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:06.884660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:06.884661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:06.884870 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:06.884908 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:06.884923 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:06.885011 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:06.885140 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:07:06.885170 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260808 10:07:06.886533 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:06.887004 23165 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } }
I20260808 10:07:06.887194 23165 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:06.887230 23165 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1529509bad4f8d964523a97195bc73, State: Initialized, Role: FOLLOWER
I20260808 10:07:06.887354 23165 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } }
I20260808 10:07:06.887442 23165 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:06.887471 23165 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:06.887517 23165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:06.888290 23165 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } }
I20260808 10:07:06.888414 23165 leader_election.cc:302] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec1529509bad4f8d964523a97195bc73; no voters:
I20260808 10:07:06.888597 23165 leader_election.cc:288] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:06.888641 23180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:06.888797 23180 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [term 1 LEADER]: Becoming Leader. State: Replica: ec1529509bad4f8d964523a97195bc73, State: Running, Role: LEADER
I20260808 10:07:06.888855 23165 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:06.888963 23180 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } }
I20260808 10:07:06.889286 23180 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1529509bad4f8d964523a97195bc73. Latest consensus state: current_term: 1 leader_uuid: "ec1529509bad4f8d964523a97195bc73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } } }
I20260808 10:07:06.889384 23180 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:06.889298 23181 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec1529509bad4f8d964523a97195bc73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1529509bad4f8d964523a97195bc73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33871 } } }
I20260808 10:07:06.889551 23181 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73 [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:06.890847 23207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:06.890902 23207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:06.890983 23194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:06.891175 23194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:06.891688 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:06.888165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:44359 (local address 127.0.0.1:33871)
0808 10:07:06.888224 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:06.888226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:06.888332 (+ 106us) server_negotiation.cc:407] Connection header received
0808 10:07:06.888397 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:06.888398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:06.888409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:06.888457 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:06.888579 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:06.889276 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:06.890185 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:06.890489 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:06.890646 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:06.891372 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:06.891373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:06.891374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:06.891378 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:06.891391 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:06.891396 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:06.891446 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:06.891603 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:07:06.891613 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:07:06.892879 23194 catalog_manager.cc:1347] Generated new cluster ID: b1bec4eb34314630a6764dbdcd2e9fe8
I20260808 10:07:06.892922 23194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:06.916352 23194 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:06.916887 23194 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:06.921283 23194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec1529509bad4f8d964523a97195bc73: Generated new TSK 0
I20260808 10:07:06.921468 23194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:06.929142 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23074
W20260808 10:07:06.939446 23051 connection.cc:535] client connection to 127.0.0.1:33871 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33871): Connection reset by peer (error 104)
2026-08-08T10:07:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:06 dist-test-slave-nfb5 krb5kdc[23214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:06 dist-test-slave-nfb5 krb5kdc[23214](info): set up 2 sockets
Aug 08 10:07:06 dist-test-slave-nfb5 krb5kdc[23214](info): commencing operation
krb5kdc: starting...
W20260808 10:07:08.149621 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:08 dist-test-slave-nfb5 krb5kdc[23214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:08.281306 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:08.363776 23234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:08.363987 23234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:08.364054 23234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:08.365187 23234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:08.365232 23234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:08.365247 23234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:08.365260 23234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:08.365273 23234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:08.366779 23234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:08.367038 23234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:08.367229 23234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:08.372092 23241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:08.372089 23242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:08.372167 23234 server_base.cc:1034] running on GCE node
W20260808 10:07:08.372089 23244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:08.372526 23234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:08.372866 23234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:08.374043 23234 hybrid_clock.cc:648] HybridClock initialized: now 1786183628374026 us; error 48 us; skew 500 ppm
Aug 08 10:07:08 dist-test-slave-nfb5 krb5kdc[23214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:08.376893 23234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:08.377342 23234 webserver.cc:460] Webserver started at http://127.0.0.1:35515/ using document root <none> and password file <none>
I20260808 10:07:08.377593 23234 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:08.377640 23234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:08.377799 23234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:08.378806 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "32e8094338134038a438fb6d20ed5bf8"
format_stamp: "Formatted at 2026-08-08 10:07:08 on dist-test-slave-nfb5"
server_key: "64433c2e29d6dfab2fe4d05f1dbc035b"
server_key_iv: "00bbd174a2fac50ecbbfc66a264c2167"
server_key_version: "encryptionkey@0"
I20260808 10:07:08.379235 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "32e8094338134038a438fb6d20ed5bf8"
format_stamp: "Formatted at 2026-08-08 10:07:08 on dist-test-slave-nfb5"
server_key: "64433c2e29d6dfab2fe4d05f1dbc035b"
server_key_iv: "00bbd174a2fac50ecbbfc66a264c2167"
server_key_version: "encryptionkey@0"
I20260808 10:07:08.380576 23234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:08.381349 23252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:08.381528 23234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:08.381630 23234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "32e8094338134038a438fb6d20ed5bf8"
format_stamp: "Formatted at 2026-08-08 10:07:08 on dist-test-slave-nfb5"
server_key: "64433c2e29d6dfab2fe4d05f1dbc035b"
server_key_iv: "00bbd174a2fac50ecbbfc66a264c2167"
server_key_version: "encryptionkey@0"
I20260808 10:07:08.381711 23234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:08.385001 23234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:08.387121 23234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:08.387286 23234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:08.391363 23234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40015
I20260808 10:07:08.391419 23320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40015 every 8 connection(s)
I20260808 10:07:08.391768 23234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:08.392784 23321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:08.395114 23321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: Bootstrap starting.
I20260808 10:07:08.395699 23321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:08.396008 23321 log.cc:824] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:08.396632 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23234
I20260808 10:07:08.396757 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:08.396852 23321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: No bootstrap required, opened a new log
I20260808 10:07:08.396940 25226 external_mini_cluster.cc:1428] Setting key 4e69160403fcf58105cefa7537962971
I20260808 10:07:08.398653 23321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } }
I20260808 10:07:08.398826 23321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:08.398936 23321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e8094338134038a438fb6d20ed5bf8, State: Initialized, Role: FOLLOWER
I20260808 10:07:08.399125 23321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } }
I20260808 10:07:08.399236 23321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:08.399274 23321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:08.399331 23321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:08.400031 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:08.397484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:39655 (local address 127.0.0.1:40015)
0808 10:07:08.397683 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:08.397693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:08.397728 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:07:08.398638 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:08.398645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:08.398671 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:08.398820 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:08.399587 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:08.399590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:08.399591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:08.399595 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:08.399646 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:08.399658 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:08.399738 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:08.399808 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:07:08.399850 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260808 10:07:08.400393 23321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } }
I20260808 10:07:08.400516 23321 leader_election.cc:302] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32e8094338134038a438fb6d20ed5bf8; no voters:
I20260808 10:07:08.400696 23321 leader_election.cc:288] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:08.400725 23326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:08.400949 23326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [term 1 LEADER]: Becoming Leader. State: Replica: 32e8094338134038a438fb6d20ed5bf8, State: Running, Role: LEADER
I20260808 10:07:08.400969 23321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:08.401051 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:08.401166 23326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } }
I20260808 10:07:08.401913 23328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32e8094338134038a438fb6d20ed5bf8. Latest consensus state: current_term: 1 leader_uuid: "32e8094338134038a438fb6d20ed5bf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } } }
I20260808 10:07:08.402009 23328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:08.402669 23327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32e8094338134038a438fb6d20ed5bf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e8094338134038a438fb6d20ed5bf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40015 } } }
W20260808 10:07:08.403139 23361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 10:07:08.403148 23327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8 [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:08.403266 23361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:08.403333 23341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:08.403517 23341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:08.404379 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:08.402993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:47165 (local address 127.0.0.1:40015)
0808 10:07:08.403028 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:08.403031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:08.403265 (+ 234us) server_negotiation.cc:407] Connection header received
0808 10:07:08.403390 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:08.403391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:08.403397 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:08.403427 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:08.404125 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:08.404126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:08.404127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:08.404132 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:08.404147 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:08.404151 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:08.404183 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:08.404286 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:07:08.404300 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:07:08.405113 23341 catalog_manager.cc:1347] Generated new cluster ID: 16d8c44af64f4b148ebd1392ace8084e
I20260808 10:07:08.405164 23341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:08.411927 23341 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:08.412083 23341 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:08.417474 23341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32e8094338134038a438fb6d20ed5bf8: Generated new TSK 0
I20260808 10:07:08.417680 23341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:08.427143 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23234
2026-08-08T10:07:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:08 dist-test-slave-nfb5 krb5kdc[23370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:08 dist-test-slave-nfb5 krb5kdc[23370](info): set up 2 sockets
Aug 08 10:07:08 dist-test-slave-nfb5 krb5kdc[23370](info): commencing operation
krb5kdc: starting...
W20260808 10:07:09.644958 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:09 dist-test-slave-nfb5 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:09.770833 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:09.854259 23388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:09.854471 23388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:09.854494 23388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:09.855757 23388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:09.855811 23388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:09.855827 23388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:09.855840 23388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:09.855854 23388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:09.857362 23388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34301
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:09.857623 23388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:09.857865 23388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:09.862809 23396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:09.862846 23388 server_base.cc:1034] running on GCE node
W20260808 10:07:09.862795 23395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:09.862861 23398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:09.863348 23388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:09.863668 23388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:09.864838 23388 hybrid_clock.cc:648] HybridClock initialized: now 1786183629864793 us; error 61 us; skew 500 ppm
Aug 08 10:07:09 dist-test-slave-nfb5 krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:09.867892 23388 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:09.868439 23388 webserver.cc:460] Webserver started at http://127.0.0.1:35087/ using document root <none> and password file <none>
I20260808 10:07:09.868707 23388 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:09.868757 23388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:09.868909 23388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:09.870016 23388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "624f8fc5969841c38dbe664aaf4bd352"
format_stamp: "Formatted at 2026-08-08 10:07:09 on dist-test-slave-nfb5"
server_key: "b43fa7674a426fd35814c34739deb769"
server_key_iv: "f2ac2e62c94e6e4ec10c14480093b76a"
server_key_version: "encryptionkey@0"
I20260808 10:07:09.870381 23388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "624f8fc5969841c38dbe664aaf4bd352"
format_stamp: "Formatted at 2026-08-08 10:07:09 on dist-test-slave-nfb5"
server_key: "b43fa7674a426fd35814c34739deb769"
server_key_iv: "f2ac2e62c94e6e4ec10c14480093b76a"
server_key_version: "encryptionkey@0"
I20260808 10:07:09.871830 23388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:09.872613 23407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:09.872898 23388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:09.872987 23388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "624f8fc5969841c38dbe664aaf4bd352"
format_stamp: "Formatted at 2026-08-08 10:07:09 on dist-test-slave-nfb5"
server_key: "b43fa7674a426fd35814c34739deb769"
server_key_iv: "f2ac2e62c94e6e4ec10c14480093b76a"
server_key_version: "encryptionkey@0"
I20260808 10:07:09.873040 23388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:09.891446 23388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:09.893589 23388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:09.893707 23388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:09.898008 23388 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34301
I20260808 10:07:09.898068 23473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34301 every 8 connection(s)
I20260808 10:07:09.898438 23388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:09.899531 23476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:09.901855 23476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352: Bootstrap starting.
I20260808 10:07:09.902464 23476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:09.902773 23476 log.cc:824] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:09.903644 23476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352: No bootstrap required, opened a new log
I20260808 10:07:09.905369 23476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260808 10:07:09.905539 23476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:09.905588 23476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624f8fc5969841c38dbe664aaf4bd352, State: Initialized, Role: FOLLOWER
I20260808 10:07:09.905735 23476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260808 10:07:09.905821 23476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:09.905866 23476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:09.905915 23476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:09.906474 23476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260808 10:07:09.906606 23476 leader_election.cc:302] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 624f8fc5969841c38dbe664aaf4bd352; no voters:
I20260808 10:07:09.906646 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23388
I20260808 10:07:09.906744 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:09.906814 23476 leader_election.cc:288] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:09.906886 23482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:09.906973 25226 external_mini_cluster.cc:1428] Setting key 9e158d4d606845f9723ee96d13f49d43
I20260808 10:07:09.907102 23476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:09.907131 23482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [term 1 LEADER]: Becoming Leader. State: Replica: 624f8fc5969841c38dbe664aaf4bd352, State: Running, Role: LEADER
I20260808 10:07:09.907634 23482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260808 10:07:09.908249 23483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "624f8fc5969841c38dbe664aaf4bd352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } } }
I20260808 10:07:09.908337 23484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 624f8fc5969841c38dbe664aaf4bd352. Latest consensus state: current_term: 1 leader_uuid: "624f8fc5969841c38dbe664aaf4bd352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624f8fc5969841c38dbe664aaf4bd352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } } }
I20260808 10:07:09.908452 23484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:09.908356 23483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:09.908941 23495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:09.909694 23495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:09.911347 23495 catalog_manager.cc:1347] Generated new cluster ID: 558b421e4d9f4ed1958be01c821031c1
I20260808 10:07:09.911404 23495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:09 dist-test-slave-nfb5 krb5kdc[23370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:07:09.918494 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:09.907477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:37727 (local address 127.0.0.1:34301)
0808 10:07:09.908040 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:09.908050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:09.908097 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:07:09.908234 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:09.908242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:09.908303 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:09.908509 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:09.909392 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:09.910383 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:09.911511 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:09.911931 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:09.912061 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:09.914503 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:09.914507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:09.914507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:09.914518 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:09.916851 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:09.917276 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:09.917280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:09.917280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:09.917383 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:09.917630 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:09.917631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:09.917632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:09.917884 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:09.918085 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:09.918185 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:07:09.918291 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":92,"threads_started":1}
I20260808 10:07:09.919582 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:09.925290 23495 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:09.925971 23495 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:09.927445 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:09.920953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:40445 (local address 127.0.0.1:34301)
0808 10:07:09.920991 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:09.920993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:09.921114 (+ 121us) server_negotiation.cc:407] Connection header received
0808 10:07:09.921192 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:09.921193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:09.921203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:09.921265 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:09.921776 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:09.922264 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:09.923013 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:09.923253 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:09.923335 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:09.924536 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:09.924537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:09.924538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:09.924545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:09.926168 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:09.926540 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:09.926541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:09.926541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:09.926598 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:09.926840 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:09.926842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:09.926842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:09.927033 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:09.927165 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:09.927275 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 10:07:09.927338 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:07:09.932623 23495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 624f8fc5969841c38dbe664aaf4bd352: Generated new TSK 0
I20260808 10:07:09.932816 23495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:09.938596 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23388
W20260808 10:07:09.952446 23367 connection.cc:535] client connection to 127.0.0.1:34301 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34301): Connection reset by peer (error 104)
2026-08-08T10:07:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:10 dist-test-slave-nfb5 krb5kdc[23516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:10 dist-test-slave-nfb5 krb5kdc[23516](info): set up 2 sockets
Aug 08 10:07:10 dist-test-slave-nfb5 krb5kdc[23516](info): commencing operation
krb5kdc: starting...
W20260808 10:07:11.169991 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:11 dist-test-slave-nfb5 krb5kdc[23516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:11.296343 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:11.377791 23534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:11.378000 23534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:11.378022 23534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:11.379097 23534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:11.379133 23534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:11.379148 23534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:11.379160 23534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:11.379172 23534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:11.380604 23534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:11.380839 23534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:11.381099 23534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:11.386044 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:11.386150 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:11.386229 23534 server_base.cc:1034] running on GCE node
W20260808 10:07:11.386161 23544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:11.386637 23534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:11.386981 23534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:11.388130 23534 hybrid_clock.cc:648] HybridClock initialized: now 1786183631388109 us; error 45 us; skew 500 ppm
Aug 08 10:07:11 dist-test-slave-nfb5 krb5kdc[23516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183631, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:11.394479 23534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:11.394888 23534 webserver.cc:460] Webserver started at http://127.0.0.1:34041/ using document root <none> and password file <none>
I20260808 10:07:11.395138 23534 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:11.395183 23534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:11.395275 23534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:11.396296 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "3761f2ea575c41deaa20328bd656876a"
format_stamp: "Formatted at 2026-08-08 10:07:11 on dist-test-slave-nfb5"
server_key: "d1d56d48243034ffec9db724f4e80531"
server_key_iv: "a684735f8af07b0233ed7b0896362ade"
server_key_version: "encryptionkey@0"
I20260808 10:07:11.396615 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "3761f2ea575c41deaa20328bd656876a"
format_stamp: "Formatted at 2026-08-08 10:07:11 on dist-test-slave-nfb5"
server_key: "d1d56d48243034ffec9db724f4e80531"
server_key_iv: "a684735f8af07b0233ed7b0896362ade"
server_key_version: "encryptionkey@0"
I20260808 10:07:11.397884 23534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:11.398586 23556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:11.398787 23534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:11.398880 23534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "3761f2ea575c41deaa20328bd656876a"
format_stamp: "Formatted at 2026-08-08 10:07:11 on dist-test-slave-nfb5"
server_key: "d1d56d48243034ffec9db724f4e80531"
server_key_iv: "a684735f8af07b0233ed7b0896362ade"
server_key_version: "encryptionkey@0"
I20260808 10:07:11.398985 23534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:11.418452 23534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:11.420775 23534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:11.420892 23534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:11.424875 23534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35823
I20260808 10:07:11.424952 23626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35823 every 8 connection(s)
I20260808 10:07:11.425303 23534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:11.426221 23627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:11.428565 23627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a: Bootstrap starting.
I20260808 10:07:11.429178 23627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:11.429494 23627 log.cc:824] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:11.430212 23627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a: No bootstrap required, opened a new log
I20260808 10:07:11.431628 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23534
I20260808 10:07:11.431727 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:11.431936 25226 external_mini_cluster.cc:1428] Setting key fbff47620e1a1ed5c6b79d0edec22f1b
I20260808 10:07:11.432152 23627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260808 10:07:11.432343 23627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:11.432447 23627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3761f2ea575c41deaa20328bd656876a, State: Initialized, Role: FOLLOWER
I20260808 10:07:11.432632 23627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260808 10:07:11.432766 23627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:11.432848 23627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:11.432905 23627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:11.433946 23627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260808 10:07:11.434075 23627 leader_election.cc:302] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3761f2ea575c41deaa20328bd656876a; no voters:
I20260808 10:07:11.434247 23627 leader_election.cc:288] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:11.434271 23634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:11.434396 23634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [term 1 LEADER]: Becoming Leader. State: Replica: 3761f2ea575c41deaa20328bd656876a, State: Running, Role: LEADER
I20260808 10:07:11.434515 23627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:11.434540 23634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260808 10:07:11.434998 23634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3761f2ea575c41deaa20328bd656876a. Latest consensus state: current_term: 1 leader_uuid: "3761f2ea575c41deaa20328bd656876a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } } }
I20260808 10:07:11.435263 23634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:11.435177 23635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3761f2ea575c41deaa20328bd656876a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3761f2ea575c41deaa20328bd656876a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } } }
I20260808 10:07:11.435717 23635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:11.435868 23647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:11.436518 23647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:11.437969 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:11.432527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:57889 (local address 127.0.0.1:35823)
0808 10:07:11.432767 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:11.432776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:11.432810 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:11.432940 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:11.432947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:11.433003 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:11.433122 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:11.433395 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:11.434469 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:11.435665 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:11.436281 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:11.436425 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:11.437175 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:11.437179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:11.437180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:11.437396 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:11.437437 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:11.437452 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:11.437583 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:11.437714 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 10:07:11.437743 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":127,"threads_started":1}
I20260808 10:07:11.438398 23647 catalog_manager.cc:1347] Generated new cluster ID: e93247fa28934ab9b026290bb6cb7f0c
I20260808 10:07:11.438449 23647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:11.439031 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:11.444023 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:11.440442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:32855 (local address 127.0.0.1:35823)
0808 10:07:11.440473 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:11.440475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:11.440706 (+ 231us) server_negotiation.cc:407] Connection header received
0808 10:07:11.440784 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:11.440785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:11.440795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:11.440821 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:11.440977 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:11.441720 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:11.442687 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:11.443010 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:11.443048 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:11.443783 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:11.443785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:11.443785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:11.443789 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:11.443801 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:11.443805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:11.443834 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:11.443942 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:07:11.443953 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 10:07:11.446296 23647 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:11.447013 23647 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:11.455982 23647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3761f2ea575c41deaa20328bd656876a: Generated new TSK 0
I20260808 10:07:11.456157 23647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:11.458648 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23534
W20260808 10:07:11.471190 23514 connection.cc:535] client connection to 127.0.0.1:35823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35823): Connection reset by peer (error 104)
2026-08-08T10:07:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:11 dist-test-slave-nfb5 krb5kdc[23671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:11 dist-test-slave-nfb5 krb5kdc[23671](info): set up 2 sockets
Aug 08 10:07:11 dist-test-slave-nfb5 krb5kdc[23671](info): commencing operation
krb5kdc: starting...
W20260808 10:07:12.674228 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:12 dist-test-slave-nfb5 krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:07:12.805622 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:12.890566 23692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:12.890781 23692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:12.890805 23692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:12.891918 23692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:12.891956 23692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:12.891973 23692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:12.891985 23692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:12.891999 23692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:12.893440 23692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45735
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:12.893643 23692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:12.893834 23692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:12.898864 23692 server_base.cc:1034] running on GCE node
W20260808 10:07:12.898824 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:12.898825 23700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:12.898842 23699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:12.899336 23692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:12.899592 23692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:12.900767 23692 hybrid_clock.cc:648] HybridClock initialized: now 1786183632900735 us; error 46 us; skew 500 ppm
Aug 08 10:07:12 dist-test-slave-nfb5 krb5kdc[23671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:12.903216 23692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:07:12.903707 23692 webserver.cc:460] Webserver started at http://127.0.0.1:46329/ using document root <none> and password file <none>
I20260808 10:07:12.903942 23692 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:12.904016 23692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:12.904168 23692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:12.905254 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "11672abd080b48e1a2e408eed6678e1d"
format_stamp: "Formatted at 2026-08-08 10:07:12 on dist-test-slave-nfb5"
server_key: "1869a7b52f33163f115e78861f6110a7"
server_key_iv: "8e89f7f4e256b34c25652bba2a3fc73d"
server_key_version: "encryptionkey@0"
I20260808 10:07:12.905633 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "11672abd080b48e1a2e408eed6678e1d"
format_stamp: "Formatted at 2026-08-08 10:07:12 on dist-test-slave-nfb5"
server_key: "1869a7b52f33163f115e78861f6110a7"
server_key_iv: "8e89f7f4e256b34c25652bba2a3fc73d"
server_key_version: "encryptionkey@0"
I20260808 10:07:12.907135 23692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:12.907924 23711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:12.908133 23692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:12.908249 23692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "11672abd080b48e1a2e408eed6678e1d"
format_stamp: "Formatted at 2026-08-08 10:07:12 on dist-test-slave-nfb5"
server_key: "1869a7b52f33163f115e78861f6110a7"
server_key_iv: "8e89f7f4e256b34c25652bba2a3fc73d"
server_key_version: "encryptionkey@0"
I20260808 10:07:12.908325 23692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:12.911656 23692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:12.913656 23692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:12.913823 23692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:12.918131 23692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45735
I20260808 10:07:12.918166 23775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45735 every 8 connection(s)
I20260808 10:07:12.918512 23692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:12.919569 23776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:12.921029 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23692
I20260808 10:07:12.921155 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:12.921332 25226 external_mini_cluster.cc:1428] Setting key 32438d9f05193c153b7452ac354b3a8d
I20260808 10:07:12.922349 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d: Bootstrap starting.
I20260808 10:07:12.923072 23776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:12.923491 23776 log.cc:824] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:12.923921 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:12.921889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:48577 (local address 127.0.0.1:45735)
0808 10:07:12.922084 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:12.922093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:12.922131 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:07:12.922604 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:12.922611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:12.922637 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:12.922763 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:12.923516 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:12.923519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:12.923519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:12.923523 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:12.923557 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:12.923567 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:12.923697 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:12.923729 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 10:07:12.923772 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260808 10:07:12.924307 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d: No bootstrap required, opened a new log
I20260808 10:07:12.924926 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:12.926954 23776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } }
I20260808 10:07:12.927135 23776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:12.927177 23776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11672abd080b48e1a2e408eed6678e1d, State: Initialized, Role: FOLLOWER
I20260808 10:07:12.927312 23776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } }
I20260808 10:07:12.927423 23776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:12.927467 23776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:12.927520 23776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:12.927785 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:12.926613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.108:53335 (local address 127.0.0.1:45735)
0808 10:07:12.926653 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:12.926655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:12.926727 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:07:12.926800 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:12.926800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:12.926804 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:12.926866 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:12.927558 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:12.927559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:12.927559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:12.927564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:12.927578 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:12.927582 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:12.927635 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:12.927693 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:07:12.927708 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:07:12.928194 23776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } }
I20260808 10:07:12.928315 23776 leader_election.cc:302] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11672abd080b48e1a2e408eed6678e1d; no voters:
I20260808 10:07:12.928491 23776 leader_election.cc:288] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:12.928514 23787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:12.928679 23787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [term 1 LEADER]: Becoming Leader. State: Replica: 11672abd080b48e1a2e408eed6678e1d, State: Running, Role: LEADER
I20260808 10:07:12.928746 23776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:12.928830 23787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } }
I20260808 10:07:12.929227 23788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11672abd080b48e1a2e408eed6678e1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } } }
I20260808 10:07:12.929339 23788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:12.929221 23789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11672abd080b48e1a2e408eed6678e1d. Latest consensus state: current_term: 1 leader_uuid: "11672abd080b48e1a2e408eed6678e1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11672abd080b48e1a2e408eed6678e1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45735 } } }
I20260808 10:07:12.929813 23789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:12.929569 23802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:12.930274 23802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:12.932077 23802 catalog_manager.cc:1347] Generated new cluster ID: be4dc95ae20a40209277688547efd4f0
I20260808 10:07:12.932130 23802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:12.941932 23802 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:12.942094 23802 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:12.948743 23802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11672abd080b48e1a2e408eed6678e1d: Generated new TSK 0
I20260808 10:07:12.948925 23802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:12.954296 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23692
2026-08-08T10:07:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:13 dist-test-slave-nfb5 krb5kdc[23822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:13 dist-test-slave-nfb5 krb5kdc[23822](info): set up 2 sockets
Aug 08 10:07:13 dist-test-slave-nfb5 krb5kdc[23822](info): commencing operation
krb5kdc: starting...
W20260808 10:07:14.185036 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:14.314329 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40131
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:14.397837 23839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:14.398147 23839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:14.398188 23839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:14.399334 23839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:14.399399 23839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:14.399423 23839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:14.399444 23839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:14.399471 23839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:14.400995 23839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:40131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:14.401273 23839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:14.401499 23839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:14.406517 23851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:14.406522 23847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:14.406594 23839 server_base.cc:1034] running on GCE node
W20260808 10:07:14.406530 23848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:14.407001 23839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:14.407330 23839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:14.408476 23839 hybrid_clock.cc:648] HybridClock initialized: now 1786183634408456 us; error 39 us; skew 500 ppm
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183634, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:14.411108 23839 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:14.411578 23839 webserver.cc:460] Webserver started at http://127.24.162.190:34445/ using document root <none> and password file <none>
I20260808 10:07:14.411849 23839 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:14.411897 23839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:14.412057 23839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:14.413084 23839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "df5ca70cc07a4213b2f39f53aa499328"
format_stamp: "Formatted at 2026-08-08 10:07:14 on dist-test-slave-nfb5"
server_key: "24def66fbdcd97d16622db68a18ea323"
server_key_iv: "f3f3ec64dbaf8df4459ea26de8ef0d18"
server_key_version: "encryptionkey@0"
I20260808 10:07:14.413458 23839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "df5ca70cc07a4213b2f39f53aa499328"
format_stamp: "Formatted at 2026-08-08 10:07:14 on dist-test-slave-nfb5"
server_key: "24def66fbdcd97d16622db68a18ea323"
server_key_iv: "f3f3ec64dbaf8df4459ea26de8ef0d18"
server_key_version: "encryptionkey@0"
I20260808 10:07:14.414762 23839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:14.415581 23861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:14.415766 23839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:14.415876 23839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "df5ca70cc07a4213b2f39f53aa499328"
format_stamp: "Formatted at 2026-08-08 10:07:14 on dist-test-slave-nfb5"
server_key: "24def66fbdcd97d16622db68a18ea323"
server_key_iv: "f3f3ec64dbaf8df4459ea26de8ef0d18"
server_key_version: "encryptionkey@0"
I20260808 10:07:14.415954 23839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:14.449110 23839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:14.451301 23839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:14.451488 23839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:14.455870 23839 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:40131
I20260808 10:07:14.455924 23934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:40131 every 8 connection(s)
I20260808 10:07:14.456315 23839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:14.457222 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:14.459448 23936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328: Bootstrap starting.
I20260808 10:07:14.459981 23936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:14.460181 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23839
I20260808 10:07:14.460297 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:14.460330 23936 log.cc:824] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:14.460490 25226 external_mini_cluster.cc:1428] Setting key 0ef4dc4597e7bdfb4c08f1428ba48909
I20260808 10:07:14.461199 23936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328: No bootstrap required, opened a new log
I20260808 10:07:14.463486 23936 raft_consensus.cc:348] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } }
I20260808 10:07:14.463647 23936 raft_consensus.cc:374] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:14.463769 23936 raft_consensus.cc:729] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df5ca70cc07a4213b2f39f53aa499328, State: Initialized, Role: FOLLOWER
I20260808 10:07:14.463914 23936 consensus_queue.cc:260] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } }
I20260808 10:07:14.464017 23936 raft_consensus.cc:388] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:14.464063 23936 raft_consensus.cc:482] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:14.464111 23936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:14.464679 23936 raft_consensus.cc:504] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } }
I20260808 10:07:14.464835 23936 leader_election.cc:302] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df5ca70cc07a4213b2f39f53aa499328; no voters:
I20260808 10:07:14.465047 23936 leader_election.cc:288] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:14.465070 23944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:14.465289 23944 raft_consensus.cc:686] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [term 1 LEADER]: Becoming Leader. State: Replica: df5ca70cc07a4213b2f39f53aa499328, State: Running, Role: LEADER
I20260808 10:07:14.465437 23944 consensus_queue.cc:237] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } }
I20260808 10:07:14.465762 23936 sys_catalog.cc:564] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:14.465787 23944 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df5ca70cc07a4213b2f39f53aa499328. Latest consensus state: current_term: 1 leader_uuid: "df5ca70cc07a4213b2f39f53aa499328" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } } }
I20260808 10:07:14.465888 23944 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:14.466116 23951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:14.466697 23951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:14.467152 23946 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df5ca70cc07a4213b2f39f53aa499328" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ca70cc07a4213b2f39f53aa499328" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 40131 } } }
I20260808 10:07:14.467293 23946 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328 [sys.catalog]: This master's current role is: LEADER
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:14.468441 23951 catalog_manager.cc:1347] Generated new cluster ID: a952b62cbc724b7aa80d040a0c372e88
I20260808 10:07:14.468489 23951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:14.472035 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:14.461084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34601 (local address 127.24.162.190:40131)
0808 10:07:14.461251 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:14.461262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:14.461296 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:14.461706 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:14.461713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:14.461757 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:14.461921 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:14.462571 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:14.463516 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:14.464466 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:14.464952 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:14.465059 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:14.468311 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:14.468314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:14.468314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:14.468326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:14.470287 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:14.470769 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:14.470771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:14.470772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:14.470868 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:14.471161 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:14.471162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:14.471162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:14.471440 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:14.471595 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:14.471731 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 10:07:14.471832 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260808 10:07:14.473117 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:14.481478 23951 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:14.481972 23951 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:14.482748 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:14.474754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49661 (local address 127.24.162.190:40131)
0808 10:07:14.474799 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:14.474801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:14.474938 (+ 137us) server_negotiation.cc:407] Connection header received
0808 10:07:14.475158 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:14.475159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:14.475178 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:14.475235 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:14.475932 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:14.476689 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:14.477743 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:14.478097 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:14.478155 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:14.479755 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:14.479756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:14.479757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:14.479764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:14.481391 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:14.481842 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:14.481843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:14.481843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:14.481904 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:14.482135 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:14.482136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:14.482136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:14.482269 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:14.482348 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:14.482587 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 10:07:14.482656 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:07:14.495859 23951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df5ca70cc07a4213b2f39f53aa499328: Generated new TSK 0
I20260808 10:07:14.496105 23951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:14.500083 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23839
W20260808 10:07:14.513204 23817 connection.cc:535] client connection to 127.24.162.190:40131 recv error: Network error: failed to read from TLS socket (remote: 127.24.162.190:40131): Connection reset by peer (error 104)
2026-08-08T10:07:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23977](info): set up 2 sockets
Aug 08 10:07:14 dist-test-slave-nfb5 krb5kdc[23977](info): commencing operation
krb5kdc: starting...
W20260808 10:07:15.729009 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:15 dist-test-slave-nfb5 krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:15.858870 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37009
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:36129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:37009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:15.941871 23995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:15.942065 23995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:15.942087 23995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:15.943259 23995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:15.943300 23995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:15.943315 23995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:15.943329 23995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:15.943389 23995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:15.944839 23995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:36129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:37009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:37009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:15.945083 23995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:15.945348 23995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:15.950181 24003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:15.950237 24004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:15.950364 23995 server_base.cc:1034] running on GCE node
W20260808 10:07:15.950237 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:15.950722 23995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:15.951062 23995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:15.952224 23995 hybrid_clock.cc:648] HybridClock initialized: now 1786183635952209 us; error 33 us; skew 500 ppm
Aug 08 10:07:15 dist-test-slave-nfb5 krb5kdc[23977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183635, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:15.954793 23995 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:15.955241 23995 webserver.cc:460] Webserver started at http://127.24.162.190:45573/ using document root <none> and password file <none>
I20260808 10:07:15.955489 23995 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:15.955536 23995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:15.955708 23995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:15.956712 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "056edbcda53e4ebfa17550932f00d341"
format_stamp: "Formatted at 2026-08-08 10:07:15 on dist-test-slave-nfb5"
server_key: "7a104af3fce779fce1bc98a7470c5804"
server_key_iv: "d3e28c287b7b1f12fb9d3a717cdfdfe4"
server_key_version: "encryptionkey@0"
I20260808 10:07:15.957067 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "056edbcda53e4ebfa17550932f00d341"
format_stamp: "Formatted at 2026-08-08 10:07:15 on dist-test-slave-nfb5"
server_key: "7a104af3fce779fce1bc98a7470c5804"
server_key_iv: "d3e28c287b7b1f12fb9d3a717cdfdfe4"
server_key_version: "encryptionkey@0"
I20260808 10:07:15.958441 23995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:15.959267 24016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:15.959448 23995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:15.959556 23995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "056edbcda53e4ebfa17550932f00d341"
format_stamp: "Formatted at 2026-08-08 10:07:15 on dist-test-slave-nfb5"
server_key: "7a104af3fce779fce1bc98a7470c5804"
server_key_iv: "d3e28c287b7b1f12fb9d3a717cdfdfe4"
server_key_version: "encryptionkey@0"
I20260808 10:07:15.959631 23995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:15.981323 23995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:15.983238 23995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:15.983424 23995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:15.987457 23995 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:37009
I20260808 10:07:15.987494 24092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:37009 every 8 connection(s)
I20260808 10:07:15.987852 23995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:15.988734 24093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:15.991137 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341: Bootstrap starting.
I20260808 10:07:15.991791 24093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:15.992094 24093 log.cc:824] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:15.992842 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341: No bootstrap required, opened a new log
I20260808 10:07:15.994546 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 23995
I20260808 10:07:15.994659 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:15.994509 24093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } }
I20260808 10:07:15.994692 24093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:15.994735 24093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056edbcda53e4ebfa17550932f00d341, State: Initialized, Role: FOLLOWER
I20260808 10:07:15.994864 25226 external_mini_cluster.cc:1428] Setting key 503a60d9d6cd53d6cb96b28d6d26722e
I20260808 10:07:15.994958 24093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } }
I20260808 10:07:15.995060 24093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:15.995088 24093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:15.995144 24093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:15.995908 24093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } }
I20260808 10:07:15.996037 24093 leader_election.cc:302] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056edbcda53e4ebfa17550932f00d341; no voters:
I20260808 10:07:15.996217 24093 leader_election.cc:288] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:15.996248 24101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:15.996423 24101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [term 1 LEADER]: Becoming Leader. State: Replica: 056edbcda53e4ebfa17550932f00d341, State: Running, Role: LEADER
I20260808 10:07:15.996487 24093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:15.996551 24101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } }
I20260808 10:07:15.996953 24102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056edbcda53e4ebfa17550932f00d341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } } }
I20260808 10:07:15.996949 24103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056edbcda53e4ebfa17550932f00d341. Latest consensus state: current_term: 1 leader_uuid: "056edbcda53e4ebfa17550932f00d341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056edbcda53e4ebfa17550932f00d341" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 37009 } } }
I20260808 10:07:15.997062 24102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:15.997067 24103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:15.997382 24111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:15.998355 24111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:16.000542 24111 catalog_manager.cc:1347] Generated new cluster ID: 6dce7c9d85cf4c3d90c7f689ea1a9fb5
I20260808 10:07:16.000615 24111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:16.001509 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:15.995533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45837 (local address 127.24.162.190:37009)
0808 10:07:15.996011 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:15.996021 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:15.996054 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:07:15.996199 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:15.996207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:15.996266 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:15.996424 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:15.996762 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:15.998277 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:15.999420 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:15.999870 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:15.999973 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:16.000723 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:16.000727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:16.000727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:16.000936 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:16.000975 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:16.000991 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:16.001071 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:16.001265 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 10:07:16.001294 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":394,"threads_started":1}
I20260808 10:07:16.002581 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:16.007545 24111 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:16.007972 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:16.004185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42169 (local address 127.24.162.190:37009)
0808 10:07:16.004225 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:16.004227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:16.004306 (+ 79us) server_negotiation.cc:407] Connection header received
0808 10:07:16.004401 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:16.004402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:16.004428 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:16.004459 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:16.004648 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:16.005416 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:16.006529 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:16.006857 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:16.006900 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:16.007708 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:16.007709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:16.007710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:16.007714 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:16.007728 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:16.007733 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:16.007801 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:16.007884 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 10:07:16.007898 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:07:16.008186 24111 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:16.011889 24111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056edbcda53e4ebfa17550932f00d341: Generated new TSK 0
I20260808 10:07:16.012065 24111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:16.016029 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 23995
W20260808 10:07:16.028301 23975 connection.cc:535] client connection to 127.24.162.190:37009 recv error: Network error: failed to read from TLS socket (remote: 127.24.162.190:37009): Connection reset by peer (error 104)
2026-08-08T10:07:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:16 dist-test-slave-nfb5 krb5kdc[24140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:16 dist-test-slave-nfb5 krb5kdc[24140](info): set up 2 sockets
Aug 08 10:07:16 dist-test-slave-nfb5 krb5kdc[24140](info): commencing operation
krb5kdc: starting...
W20260808 10:07:17.234733 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:17 dist-test-slave-nfb5 krb5kdc[24140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:17.359521 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44711
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:32835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:44711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:17.441617 24154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:17.441866 24154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:17.441907 24154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:17.443122 24154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:17.443182 24154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:17.443208 24154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:17.443241 24154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:17.443270 24154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:17.444726 24154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:32835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:44711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:44711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:17.444998 24154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:17.445232 24154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:17.450064 24161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:17.450088 24163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:17.450073 24160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:17.450376 24154 server_base.cc:1034] running on GCE node
I20260808 10:07:17.450609 24154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:17.450889 24154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:17.452083 24154 hybrid_clock.cc:648] HybridClock initialized: now 1786183637452066 us; error 35 us; skew 500 ppm
Aug 08 10:07:17 dist-test-slave-nfb5 krb5kdc[24140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183637, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:17.461249 24154 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:17.461721 24154 webserver.cc:460] Webserver started at http://127.24.162.190:34837/ using document root <none> and password file <none>
I20260808 10:07:17.461984 24154 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:17.462072 24154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:17.462231 24154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:17.463382 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "2e0b58dce5e046fc9e172f6e004ab860"
format_stamp: "Formatted at 2026-08-08 10:07:17 on dist-test-slave-nfb5"
server_key: "255811d5b295c63c9d84db83a17e1df1"
server_key_iv: "85af05ece329e94d179e248d3cfd7774"
server_key_version: "encryptionkey@0"
I20260808 10:07:17.463747 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "2e0b58dce5e046fc9e172f6e004ab860"
format_stamp: "Formatted at 2026-08-08 10:07:17 on dist-test-slave-nfb5"
server_key: "255811d5b295c63c9d84db83a17e1df1"
server_key_iv: "85af05ece329e94d179e248d3cfd7774"
server_key_version: "encryptionkey@0"
I20260808 10:07:17.465080 24154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:17.465827 24170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:17.466039 24154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:17.466148 24154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "2e0b58dce5e046fc9e172f6e004ab860"
format_stamp: "Formatted at 2026-08-08 10:07:17 on dist-test-slave-nfb5"
server_key: "255811d5b295c63c9d84db83a17e1df1"
server_key_iv: "85af05ece329e94d179e248d3cfd7774"
server_key_version: "encryptionkey@0"
I20260808 10:07:17.466238 24154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:17.469648 24154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:17.471604 24154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:17.471768 24154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:17.476118 24154 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:44711
I20260808 10:07:17.476148 24244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:44711 every 8 connection(s)
I20260808 10:07:17.476490 24154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:17.477470 24245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:17.479748 24245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860: Bootstrap starting.
I20260808 10:07:17.480276 24245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:17.480522 24245 log.cc:824] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:17.481206 24245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860: No bootstrap required, opened a new log
I20260808 10:07:17.482748 24245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } }
I20260808 10:07:17.482880 24245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:17.482903 24245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0b58dce5e046fc9e172f6e004ab860, State: Initialized, Role: FOLLOWER
I20260808 10:07:17.483111 24245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } }
I20260808 10:07:17.483197 24245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:17.483222 24245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:17.483249 24245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:17.484074 24245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } }
I20260808 10:07:17.484164 24245 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e0b58dce5e046fc9e172f6e004ab860; no voters:
I20260808 10:07:17.484325 24245 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:17.484390 24248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:17.484473 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24154
I20260808 10:07:17.484550 24248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [term 1 LEADER]: Becoming Leader. State: Replica: 2e0b58dce5e046fc9e172f6e004ab860, State: Running, Role: LEADER
I20260808 10:07:17.484578 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:17.484625 24248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } }
I20260808 10:07:17.484766 25226 external_mini_cluster.cc:1428] Setting key 0f723bff98bfec16b7aef1a98b5437db
I20260808 10:07:17.484625 24245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:17.485059 24250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e0b58dce5e046fc9e172f6e004ab860. Latest consensus state: current_term: 1 leader_uuid: "2e0b58dce5e046fc9e172f6e004ab860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } } }
I20260808 10:07:17.485136 24250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:17.485378 24249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e0b58dce5e046fc9e172f6e004ab860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e0b58dce5e046fc9e172f6e004ab860" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 44711 } } }
I20260808 10:07:17.485496 24249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:17.485656 24253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:17.486236 24253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:17.487879 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:17.485383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56963 (local address 127.24.162.190:44711)
0808 10:07:17.485567 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:17.485577 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:17.485856 (+ 279us) server_negotiation.cc:407] Connection header received
0808 10:07:17.485970 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:17.485978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:17.486011 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:17.486153 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:17.487361 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:17.487365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:17.487366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:17.487371 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:17.487414 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:17.487429 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:17.487471 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:17.487638 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:07:17.487688 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
I20260808 10:07:17.488003 24253 catalog_manager.cc:1347] Generated new cluster ID: a92f4308509f448d888444eb425ef44d
I20260808 10:07:17.488049 24253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:17.488849 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:17.492101 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:17.490245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42597 (local address 127.24.162.190:44711)
0808 10:07:17.490282 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:17.490284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:17.491093 (+ 809us) server_negotiation.cc:407] Connection header received
0808 10:07:17.491163 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:17.491163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:17.491169 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:17.491226 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:17.491866 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:17.491868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:17.491868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:17.491872 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:17.491887 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:17.491891 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:17.491937 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:17.492001 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:07:17.492017 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:07:17.506213 24253 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:17.506434 24253 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:17.513670 24253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e0b58dce5e046fc9e172f6e004ab860: Generated new TSK 0
I20260808 10:07:17.513875 24253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:17.519358 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24154
2026-08-08T10:07:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:17 dist-test-slave-nfb5 krb5kdc[24282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:17 dist-test-slave-nfb5 krb5kdc[24282](info): set up 2 sockets
Aug 08 10:07:17 dist-test-slave-nfb5 krb5kdc[24282](info): commencing operation
krb5kdc: starting...
W20260808 10:07:18.751976 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:18 dist-test-slave-nfb5 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:18.882292 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:46063
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:46063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:18.964984 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:18.965245 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:18.965284 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:18.966459 24300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:18.966540 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:18.966564 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:18.966585 24300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:18.966616 24300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:18.968106 24300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:46063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:46063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:18.968400 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:18.968636 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:18.973664 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:18.973671 24306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:18.973743 24307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:18.974036 24300 server_base.cc:1034] running on GCE node
I20260808 10:07:18.974309 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:18.974596 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:18.975770 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786183638975756 us; error 28 us; skew 500 ppm
Aug 08 10:07:18 dist-test-slave-nfb5 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183638, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:18.978411 24300 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:18.978886 24300 webserver.cc:460] Webserver started at http://127.24.162.190:40207/ using document root <none> and password file <none>
I20260808 10:07:18.979144 24300 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:18.979224 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:18.979372 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:18.980443 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "11217498f09a431fa763d4803edbcee2"
format_stamp: "Formatted at 2026-08-08 10:07:18 on dist-test-slave-nfb5"
server_key: "bfa4447d57645f95aab0d3fc98c0a28f"
server_key_iv: "3e1ce177b6ef835dadfb88de01c3ace5"
server_key_version: "encryptionkey@0"
I20260808 10:07:18.980877 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "11217498f09a431fa763d4803edbcee2"
format_stamp: "Formatted at 2026-08-08 10:07:18 on dist-test-slave-nfb5"
server_key: "bfa4447d57645f95aab0d3fc98c0a28f"
server_key_iv: "3e1ce177b6ef835dadfb88de01c3ace5"
server_key_version: "encryptionkey@0"
I20260808 10:07:18.982254 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:18.983074 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:18.983258 24300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:18.983361 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "11217498f09a431fa763d4803edbcee2"
format_stamp: "Formatted at 2026-08-08 10:07:18 on dist-test-slave-nfb5"
server_key: "bfa4447d57645f95aab0d3fc98c0a28f"
server_key_iv: "3e1ce177b6ef835dadfb88de01c3ace5"
server_key_version: "encryptionkey@0"
I20260808 10:07:18.983446 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:19.000172 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:19.002240 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:19.002414 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:19.006569 24300 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:46063
I20260808 10:07:19.006619 24394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:46063 every 8 connection(s)
I20260808 10:07:19.007025 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:19.007701 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24300
I20260808 10:07:19.007830 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:19.008008 25226 external_mini_cluster.cc:1428] Setting key 958e6e577d4e75bf809af9d6b2ea88a5
I20260808 10:07:19.008006 24398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:19.011426 24398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2: Bootstrap starting.
I20260808 10:07:19.012060 24398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:19.012373 24398 log.cc:824] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:19.013077 24398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2: No bootstrap required, opened a new log
Aug 08 10:07:19 dist-test-slave-nfb5 krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:19.015390 24398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } }
I20260808 10:07:19.015579 24398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:19.015645 24398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11217498f09a431fa763d4803edbcee2, State: Initialized, Role: FOLLOWER
I20260808 10:07:19.015805 24398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } }
I20260808 10:07:19.015889 24398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:19.015933 24398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:19.015996 24398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:19.016618 24398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } }
I20260808 10:07:19.016748 24398 leader_election.cc:302] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11217498f09a431fa763d4803edbcee2; no voters:
I20260808 10:07:19.016932 24398 leader_election.cc:288] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:19.016970 24405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:19.017150 24405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [term 1 LEADER]: Becoming Leader. State: Replica: 11217498f09a431fa763d4803edbcee2, State: Running, Role: LEADER
I20260808 10:07:19.017230 24398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:19.017310 24405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } }
I20260808 10:07:19.017711 24405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11217498f09a431fa763d4803edbcee2. Latest consensus state: current_term: 1 leader_uuid: "11217498f09a431fa763d4803edbcee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } } }
I20260808 10:07:19.017951 24405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:19.017867 24410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11217498f09a431fa763d4803edbcee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11217498f09a431fa763d4803edbcee2" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 46063 } } }
I20260808 10:07:19.018220 24410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:19.018651 24422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:19.019250 24422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:19.020026 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:19.008546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50605 (local address 127.24.162.190:46063)
0808 10:07:19.008740 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:19.008751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:19.008785 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:19.009205 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:19.009213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:19.009294 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:19.009447 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:19.010049 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:19.011431 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:19.012259 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:19.012718 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:19.012871 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:19.015597 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:19.015602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:19.015602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:19.015615 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:19.017822 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:19.018275 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:19.018277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:19.018278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:19.018386 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:19.018634 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:19.018635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:19.018635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:19.019071 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:19.019224 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:19.019685 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 10:07:19.019825 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260808 10:07:19.020730 24422 catalog_manager.cc:1347] Generated new cluster ID: a7f3ae7f8df741d0a5355427cc13acc0
I20260808 10:07:19.020807 24422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:19.021214 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:19.027032 24422 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:19.027674 24422 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:19.030066 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:19.022856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57903 (local address 127.24.162.190:46063)
0808 10:07:19.022894 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:19.022896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:19.022905 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:19.022949 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:19.022949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:19.022958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:19.023027 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:19.023613 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:19.024382 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:19.025355 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:19.025681 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:19.025757 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:19.027195 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:19.027196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:19.027196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:19.027203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:19.028767 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:19.029118 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:19.029119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:19.029119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:19.029166 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:19.029341 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:19.029342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:19.029342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:19.029445 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:19.029523 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:19.029897 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 10:07:19.029975 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:07:19.032501 24422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11217498f09a431fa763d4803edbcee2: Generated new TSK 0
I20260808 10:07:19.032701 24422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:19.037621 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24300
2026-08-08T10:07:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:19 dist-test-slave-nfb5 krb5kdc[24448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:19 dist-test-slave-nfb5 krb5kdc[24448](info): set up 2 sockets
Aug 08 10:07:19 dist-test-slave-nfb5 krb5kdc[24448](info): commencing operation
krb5kdc: starting...
W20260808 10:07:20.254084 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:20 dist-test-slave-nfb5 krb5kdc[24448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:20.376171 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35553
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:35553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:20.461615 24463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:20.461814 24463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:20.461836 24463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:20.463011 24463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:20.463052 24463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:20.463065 24463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:20.463079 24463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:20.463091 24463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:20.464986 24463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:35553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:35553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:20.465245 24463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:20.465498 24463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:20.470556 24470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:20.470522 24471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:20.470736 24463 server_base.cc:1034] running on GCE node
W20260808 10:07:20.470534 24476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:20.471061 24463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:20.471357 24463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:20.472519 24463 hybrid_clock.cc:648] HybridClock initialized: now 1786183640472513 us; error 44 us; skew 500 ppm
Aug 08 10:07:20 dist-test-slave-nfb5 krb5kdc[24448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:20.474975 24463 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:20.475476 24463 webserver.cc:460] Webserver started at http://127.24.162.190:37221/ using document root <none> and password file <none>
I20260808 10:07:20.475724 24463 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:20.475795 24463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:20.475943 24463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:20.476994 24463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a"
format_stamp: "Formatted at 2026-08-08 10:07:20 on dist-test-slave-nfb5"
server_key: "1f1cd87440943c103c4737996fce12f1"
server_key_iv: "501162f261cf1a42f4f83078e6b21539"
server_key_version: "encryptionkey@0"
I20260808 10:07:20.477363 24463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a"
format_stamp: "Formatted at 2026-08-08 10:07:20 on dist-test-slave-nfb5"
server_key: "1f1cd87440943c103c4737996fce12f1"
server_key_iv: "501162f261cf1a42f4f83078e6b21539"
server_key_version: "encryptionkey@0"
I20260808 10:07:20.478690 24463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:20.479533 24489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:20.479782 24463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:20.479877 24463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a"
format_stamp: "Formatted at 2026-08-08 10:07:20 on dist-test-slave-nfb5"
server_key: "1f1cd87440943c103c4737996fce12f1"
server_key_iv: "501162f261cf1a42f4f83078e6b21539"
server_key_version: "encryptionkey@0"
I20260808 10:07:20.479960 24463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:20.495352 24463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:20.497860 24463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:20.498044 24463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:20.502650 24463 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:35553
I20260808 10:07:20.502733 24559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:35553 every 8 connection(s)
I20260808 10:07:20.503113 24463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:20.504053 24561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:20.506479 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: Bootstrap starting.
I20260808 10:07:20.507164 24561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:20.507449 24561 log.cc:824] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:20.508293 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: No bootstrap required, opened a new log
I20260808 10:07:20.510048 24561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } }
I20260808 10:07:20.510205 24561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:20.510232 24561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e9c0e0ee0d4a9b91a11fd6017bf81a, State: Initialized, Role: FOLLOWER
I20260808 10:07:20.510344 24561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } }
I20260808 10:07:20.510453 24561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:20.510492 24561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:20.510545 24561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:20.511366 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24463
I20260808 10:07:20.511497 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:20.511530 24561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } }
I20260808 10:07:20.511654 24561 leader_election.cc:302] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76e9c0e0ee0d4a9b91a11fd6017bf81a; no voters:
I20260808 10:07:20.511735 25226 external_mini_cluster.cc:1428] Setting key 3536f25e6abe163a166d1db345e438db
I20260808 10:07:20.511854 24561 leader_election.cc:288] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:20.511902 24567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:20.512068 24567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [term 1 LEADER]: Becoming Leader. State: Replica: 76e9c0e0ee0d4a9b91a11fd6017bf81a, State: Running, Role: LEADER
I20260808 10:07:20.512192 24561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:20.512239 24567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } }
I20260808 10:07:20.512739 24569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } } }
I20260808 10:07:20.512853 24569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:20.513062 24571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76e9c0e0ee0d4a9b91a11fd6017bf81a. Latest consensus state: current_term: 1 leader_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e9c0e0ee0d4a9b91a11fd6017bf81a" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 35553 } } }
I20260808 10:07:20.513159 24571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:20.514219 24593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:20.514288 24593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:20.514389 24585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:20.514549 24585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:20.516103 24585 catalog_manager.cc:1347] Generated new cluster ID: bea4847aa4324db1acd9a405cee54fcf
I20260808 10:07:20.516150 24585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:20.519217 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:20.512228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55915 (local address 127.24.162.190:35553)
0808 10:07:20.512601 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:20.512612 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:20.513427 (+ 815us) server_negotiation.cc:407] Connection header received
0808 10:07:20.513573 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:20.513582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:20.513671 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:20.513973 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:20.514126 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:20.515722 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:20.516896 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:20.517577 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:20.517714 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:20.518435 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:20.518439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:20.518439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:20.518660 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:20.518713 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:20.518731 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:20.518809 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:20.518877 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:07:20.518971 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":74,"threads_started":1}
I20260808 10:07:20.520167 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:20.525755 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:20.521908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51845 (local address 127.24.162.190:35553)
0808 10:07:20.521952 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:20.521955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:20.521965 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:20.522022 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:20.522023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:20.522035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:20.522108 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:20.522201 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:20.523017 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:20.524172 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:20.524510 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:20.524579 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:20.525391 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:20.525392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:20.525393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:20.525397 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:20.525411 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:20.525416 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:20.525464 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:20.525633 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:07:20.525658 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:07:20.532680 24585 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:20.533247 24585 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:20.550196 24585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76e9c0e0ee0d4a9b91a11fd6017bf81a: Generated new TSK 0
I20260808 10:07:20.550391 24585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:20.558399 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24463
2026-08-08T10:07:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:20 dist-test-slave-nfb5 krb5kdc[24609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:20 dist-test-slave-nfb5 krb5kdc[24609](info): set up 2 sockets
Aug 08 10:07:20 dist-test-slave-nfb5 krb5kdc[24609](info): commencing operation
krb5kdc: starting...
W20260808 10:07:21.779978 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:21 dist-test-slave-nfb5 krb5kdc[24609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:21.903712 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33657
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:41509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:21.986189 24625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:21.986397 24625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:21.986419 24625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:21.987573 24625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:21.987617 24625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:21.987630 24625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:21.987650 24625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:21.987663 24625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:21.989076 24625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:41509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:21.989269 24625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:21.989436 24625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:21.994293 24642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:21.994303 24639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:21.994347 24633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:21.994577 24625 server_base.cc:1034] running on GCE node
I20260808 10:07:21.994858 24625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:21.995184 24625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:21.996367 24625 hybrid_clock.cc:648] HybridClock initialized: now 1786183641996359 us; error 53 us; skew 500 ppm
Aug 08 10:07:21 dist-test-slave-nfb5 krb5kdc[24609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183641, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:21.999013 24625 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:21.999491 24625 webserver.cc:460] Webserver started at http://127.24.162.190:37789/ using document root <none> and password file <none>
I20260808 10:07:21.999691 24625 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:21.999732 24625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:21.999827 24625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:22.000869 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "be6b1e43712b465e81cd51c4ca9f2c0b"
format_stamp: "Formatted at 2026-08-08 10:07:21 on dist-test-slave-nfb5"
server_key: "6bac8dbf676fd304ce4c937b1e3f86a9"
server_key_iv: "c807381480c69033b47168235f22fe55"
server_key_version: "encryptionkey@0"
I20260808 10:07:22.001158 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "be6b1e43712b465e81cd51c4ca9f2c0b"
format_stamp: "Formatted at 2026-08-08 10:07:21 on dist-test-slave-nfb5"
server_key: "6bac8dbf676fd304ce4c937b1e3f86a9"
server_key_iv: "c807381480c69033b47168235f22fe55"
server_key_version: "encryptionkey@0"
I20260808 10:07:22.002439 24625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:22.003190 24651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:22.003369 24625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:22.003463 24625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "be6b1e43712b465e81cd51c4ca9f2c0b"
format_stamp: "Formatted at 2026-08-08 10:07:21 on dist-test-slave-nfb5"
server_key: "6bac8dbf676fd304ce4c937b1e3f86a9"
server_key_iv: "c807381480c69033b47168235f22fe55"
server_key_version: "encryptionkey@0"
I20260808 10:07:22.003520 24625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:22.006632 24625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:22.008939 24625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:22.009044 24625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:22.012944 24625 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:33657
I20260808 10:07:22.012993 24719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:33657 every 8 connection(s)
I20260808 10:07:22.013290 24625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:22.014309 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:22.016722 24720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b: Bootstrap starting.
I20260808 10:07:22.017313 24720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:22.017625 24720 log.cc:824] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:22.018357 24720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b: No bootstrap required, opened a new log
I20260808 10:07:22.018359 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24625
I20260808 10:07:22.018473 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:22.018663 25226 external_mini_cluster.cc:1428] Setting key 4186a7954d45f92ee466b9513415ac83
I20260808 10:07:22.021016 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:22.019230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51431 (local address 127.24.162.190:33657)
0808 10:07:22.019394 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:22.019402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:22.019439 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:07:22.019848 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:22.019854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:22.019877 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:22.019992 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:22.020562 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:22.020565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:22.020565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:22.020568 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:22.020602 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:22.020612 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:22.020701 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:22.020751 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 10:07:22.020782 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260808 10:07:22.021466 24720 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } }
I20260808 10:07:22.021632 24720 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:22.021685 24720 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6b1e43712b465e81cd51c4ca9f2c0b, State: Initialized, Role: FOLLOWER
I20260808 10:07:22.021849 24720 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } }
I20260808 10:07:22.021953 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:22.021968 24720 raft_consensus.cc:388] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:22.022011 24720 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:22.022063 24720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:22.022595 24720 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } }
I20260808 10:07:22.022717 24720 leader_election.cc:302] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be6b1e43712b465e81cd51c4ca9f2c0b; no voters:
I20260808 10:07:22.022919 24720 leader_election.cc:288] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:22.022969 24726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:22.023154 24726 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [term 1 LEADER]: Becoming Leader. State: Replica: be6b1e43712b465e81cd51c4ca9f2c0b, State: Running, Role: LEADER
I20260808 10:07:22.023262 24720 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:22.023281 24726 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } }
I20260808 10:07:22.023627 24726 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6b1e43712b465e81cd51c4ca9f2c0b. Latest consensus state: current_term: 1 leader_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } } }
I20260808 10:07:22.023773 24726 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:22.024030 24746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:22.024608 24746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:22.024708 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:22.023576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43161 (local address 127.24.162.190:33657)
0808 10:07:22.023609 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:22.023611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:22.023620 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:22.023643 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:22.023644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:22.023658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:22.023718 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:22.024358 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:22.024359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:22.024360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:22.024364 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:22.024378 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:22.024382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:22.024429 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:22.024621 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 10:07:22.024634 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:07:22.025266 24730 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6b1e43712b465e81cd51c4ca9f2c0b" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 33657 } } }
I20260808 10:07:22.025352 24730 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:22.026296 24746 catalog_manager.cc:1347] Generated new cluster ID: b438fac5015c4d1382b3c05e8933dbbd
I20260808 10:07:22.026355 24746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:22.064664 24746 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:22.064872 24746 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:22.078675 24746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be6b1e43712b465e81cd51c4ca9f2c0b: Generated new TSK 0
I20260808 10:07:22.078938 24746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:22.084708 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24625
2026-08-08T10:07:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1527 ms)
[----------] 18 tests from AuthTokenIssuingTest (27716 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:22 dist-test-slave-nfb5 krb5kdc[24770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:22 dist-test-slave-nfb5 krb5kdc[24770](info): set up 2 sockets
Aug 08 10:07:22 dist-test-slave-nfb5 krb5kdc[24770](info): commencing operation
krb5kdc: starting...
W20260808 10:07:23.311794 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:23.437765 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45529
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:23.521250 24788 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:23.521458 24788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:23.521476 24788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:23.521497 24788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:23.522554 24788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:23.522590 24788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:23.522604 24788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:23.522617 24788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:23.522629 24788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:23.524124 24788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:23.524320 24788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:23.524487 24788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:23.529271 24795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:23.529265 24794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:23.529349 24797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:23.529798 24788 server_base.cc:1034] running on GCE node
I20260808 10:07:23.530032 24788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:23.530334 24788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:23.531503 24788 hybrid_clock.cc:648] HybridClock initialized: now 1786183643531492 us; error 31 us; skew 500 ppm
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:23.540149 24788 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:23.540627 24788 webserver.cc:460] Webserver started at http://127.24.162.190:44817/ using document root <none> and password file <none>
I20260808 10:07:23.540904 24788 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:23.540977 24788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:23.541129 24788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:23.542212 24788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "a3567fc915184a55a55ca87ed81363ff"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "09dd8ca3ec0bb791dbe1051e2823f5e7"
server_key_iv: "52a33c6257479a81d8f35b790848223f"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.542575 24788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "a3567fc915184a55a55ca87ed81363ff"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "09dd8ca3ec0bb791dbe1051e2823f5e7"
server_key_iv: "52a33c6257479a81d8f35b790848223f"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.544008 24788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:23.544751 24807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:23.545071 24788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:23.545218 24788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "a3567fc915184a55a55ca87ed81363ff"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "09dd8ca3ec0bb791dbe1051e2823f5e7"
server_key_iv: "52a33c6257479a81d8f35b790848223f"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.545302 24788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:23.567071 24788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:23.568971 24788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:23.569146 24788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:23.573629 24788 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:45529
I20260808 10:07:23.573697 24880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:45529 every 8 connection(s)
I20260808 10:07:23.574069 24788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:23.575054 24881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:23.576627 24881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:23.583416 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24788
I20260808 10:07:23.583528 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:23.583709 25226 external_mini_cluster.cc:1428] Setting key 23f7a689c6219dbbf1cb2f340209dfcd
I20260808 10:07:23.586441 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.577287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45529 (local address 127.0.0.1:42340)
0808 10:07:23.577456 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.577492 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.577952 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.578212 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.578215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.578460 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.578612 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.578615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.579722 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.579723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.580464 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.580467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.580938 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.580939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.580996 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.581517 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.581525 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.583030 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.584859 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.584868 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.584870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.585157 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.585405 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.585406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.585407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.585537 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.585998 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.586005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.586065 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.586148 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.586275 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260808 10:07:23.586520 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.577449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.24.162.190:45529)
0808 10:07:23.577572 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.577577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.577617 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:07:23.578080 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.578083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.578098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.578174 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.578726 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.579625 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.580543 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.580860 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.580936 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.583107 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.583115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.583115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.583121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.584774 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.585241 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.585244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.585245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.585337 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.585617 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.585618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.585618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.585913 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.586019 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.586210 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.586372 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260808 10:07:23.587342 24881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.587707 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.587567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40798)
0808 10:07:23.587619 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.587675 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:07:23.587864 24811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
I20260808 10:07:23.588397 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.588256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40810)
0808 10:07:23.588328 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.588362 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:07:23.588585 24881 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:34395: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.162.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
I20260808 10:07:23.610330 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:34395
--webserver_interface=127.24.162.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:07:23.637830 24881 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } attempt: 1
I20260808 10:07:23.638396 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.638187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40816)
0808 10:07:23.638268 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.638353 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:07:23.638850 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.638716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40818)
0808 10:07:23.638775 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.638815 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:07:23.639008 24881 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:34395: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
W20260808 10:07:23.694110 24890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:23.694319 24890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:23.694337 24890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:23.694357 24890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:23.695588 24890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:23.695631 24890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:23.695645 24890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:23.695658 24890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:23.695670 24890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:23.697093 24890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.162.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:23.697280 24890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:23.697450 24890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:23.702299 24896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:23.702322 24901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:23.702374 24890 server_base.cc:1034] running on GCE node
W20260808 10:07:23.702308 24897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:23.702772 24890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:23.703094 24890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:23.704276 24890 hybrid_clock.cc:648] HybridClock initialized: now 1786183643704263 us; error 31 us; skew 500 ppm
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:23.706792 24890 init.cc:377] Logged in from keytab as kudu/127.24.162.189@KRBTEST.COM (short username kudu)
I20260808 10:07:23.707160 24881 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } attempt: 2
I20260808 10:07:23.707276 24890 webserver.cc:460] Webserver started at http://127.24.162.189:36249/ using document root <none> and password file <none>
I20260808 10:07:23.707571 24890 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:23.707593 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.707442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40824)
0808 10:07:23.707511 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.707553 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:07:23.707640 24890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:23.707787 24890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:23.707989 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.707872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40834)
0808 10:07:23.707927 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.707954 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 10:07:23.708125 24881 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:34395: Network error: Client connection negotiation failed: client connection to 127.24.162.189:34395: connect: Connection refused (error 111)
I20260808 10:07:23.708914 24890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data/instance:
uuid: "7998c663048c4fc9a34c6d64d0a9908c"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "eba1f5b627f0f0038ddee80477c7c883"
server_key_iv: "e62c916436120cc11f046de8d05a280b"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.709282 24890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal/instance:
uuid: "7998c663048c4fc9a34c6d64d0a9908c"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "eba1f5b627f0f0038ddee80477c7c883"
server_key_iv: "e62c916436120cc11f046de8d05a280b"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.710642 24890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:23.711436 24910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:23.711632 24890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:23.711727 24890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "7998c663048c4fc9a34c6d64d0a9908c"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "eba1f5b627f0f0038ddee80477c7c883"
server_key_iv: "e62c916436120cc11f046de8d05a280b"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.711802 24890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:23.734428 24890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:23.736840 24890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:23.737003 24890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:23.741413 24890 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.189:34395
I20260808 10:07:23.741467 24977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.189:34395 every 8 connection(s)
I20260808 10:07:23.741853 24890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
I20260808 10:07:23.742794 24978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:23.744412 24978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.745893 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24890
I20260808 10:07:23.745993 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-1/wal/instance
I20260808 10:07:23.746171 25226 external_mini_cluster.cc:1428] Setting key c18bdf9c0ddada29a7f4c22e5dede2a9
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:23.755224 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.745141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.24.162.190:45529)
0808 10:07:23.745227 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.745229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.745410 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:07:23.745889 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.745890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.745909 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.745984 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.746752 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.747524 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.748959 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.749215 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.749296 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.751865 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.751867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.751867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.751876 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.753623 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.754052 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.754054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.754054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.754127 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.754381 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.754383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.754383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.754535 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.754616 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.754998 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.755090 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:07:23.755435 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.745090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45529 (local address 127.0.0.1:42352)
0808 10:07:23.745288 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.745325 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.745783 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.746138 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.746141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.746457 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.746653 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.746657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.747618 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.747619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.748860 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.748863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.749318 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.749318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.749420 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.750043 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.750051 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.751754 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.753733 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.753741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.753742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.753970 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.754206 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.754206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.754206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.754309 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.754642 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.754650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.754799 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.754907 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.755133 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.24.162.188@KRBTEST.COM; defaulting to no policy
I20260808 10:07:23.756098 24978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.188@KRBTEST.COM" created.
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:23.763761 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.756424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40840)
0808 10:07:23.756493 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.756505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.756552 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.756895 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.756896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.757019 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.757069 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.757071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.757917 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.757918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.758399 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.758400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.759135 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.759136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.759196 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.759647 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.759652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.760822 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.762605 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.762605 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.762606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.762777 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.763072 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.763073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.763073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.763130 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.763443 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.763444 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.763483 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.763545 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.763645 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:07:23.763845 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.756511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.24.162.189:34395)
0808 10:07:23.756692 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.756696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.756713 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:07:23.756732 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.756733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.756744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.756847 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.757135 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.757821 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.758596 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.759044 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.759132 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.760922 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.760931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.760932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.760940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.762498 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.762873 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.762876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.762876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.763002 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.763203 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.763204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.763204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.763369 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.763466 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.763583 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.763699 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260808 10:07:23.764386 24978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.764737 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.764632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47400)
0808 10:07:23.764673 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.764716 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 10:07:23.764907 24916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
I20260808 10:07:23.765333 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.765261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47414)
0808 10:07:23.765287 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.765311 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 10:07:23.765463 24978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38337: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
I20260808 10:07:23.775689 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:38337
--webserver_interface=127.24.162.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:07:23.799304 24881 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } attempt: 3
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:23.806569 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.799788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.24.162.189:34395)
0808 10:07:23.799864 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.799866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.799877 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:23.799918 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.799919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.799938 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.800002 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.800350 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.800894 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.801622 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.801853 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.801931 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.803941 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.803942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.803943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.803952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.805398 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.805766 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.805767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.805768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.805827 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.806039 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.806040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.806040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.806153 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.806230 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.806388 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.806470 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:07:23.806569 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.799691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40846)
0808 10:07:23.799774 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.799785 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.799854 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.800010 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.800011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.800185 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.800255 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.800257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.800986 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.800987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.801530 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.801532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.801944 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.801945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.802027 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.802529 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.802534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.803850 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.805501 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.805502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.805503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.805681 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.805905 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.805906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.805906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.805970 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.806223 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.806224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.806274 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.806363 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.806467 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:07:23.806941 24881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.807301 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.807199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47418)
0808 10:07:23.807249 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.807276 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 10:07:23.807753 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.807627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47422)
0808 10:07:23.807698 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.807731 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:07:23.807896 24881 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38337: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
I20260808 10:07:23.814679 24978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } attempt: 1
I20260808 10:07:23.815126 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.814984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47438)
0808 10:07:23.815038 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.815090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:07:23.815531 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.815417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47446)
0808 10:07:23.815461 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.815503 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:07:23.815650 24978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38337: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
I20260808 10:07:23.839035 24881 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } attempt: 1
I20260808 10:07:23.839504 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.839374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47460)
0808 10:07:23.839439 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.839477 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 10:07:23.839905 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.839819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47470)
0808 10:07:23.839865 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.839889 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:07:23.840029 24881 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38337: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
W20260808 10:07:23.859881 24988 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:23.860096 24988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:23.860114 24988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:23.860167 24988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:23.861333 24988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:23.861372 24988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:23.861387 24988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:23.861399 24988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:23.861452 24988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:23.862936 24988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45529,127.24.162.189:34395,127.24.162.188:38337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:38337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.162.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:23.863194 24988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:23.863430 24988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:23.868305 25000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:23.868352 24997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:23.868422 24988 server_base.cc:1034] running on GCE node
W20260808 10:07:23.868357 24996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:23.868764 24988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:23.869089 24988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:23.870260 24988 hybrid_clock.cc:648] HybridClock initialized: now 1786183643870236 us; error 37 us; skew 500 ppm
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:23.872705 24988 init.cc:377] Logged in from keytab as kudu/127.24.162.188@KRBTEST.COM (short username kudu)
I20260808 10:07:23.873178 24988 webserver.cc:460] Webserver started at http://127.24.162.188:38305/ using document root <none> and password file <none>
I20260808 10:07:23.873400 24988 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:23.873471 24988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:23.873628 24988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:23.874668 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data/instance:
uuid: "29ac9a39a9804081987566226769aa50"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "4d4d8064b18b87a842d9fb1b85ea0da4"
server_key_iv: "bff24a1928678b5803263062ba9a6de4"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.875069 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal/instance:
uuid: "29ac9a39a9804081987566226769aa50"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "4d4d8064b18b87a842d9fb1b85ea0da4"
server_key_iv: "bff24a1928678b5803263062ba9a6de4"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.876398 24988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:23.877135 25011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:23.877331 24988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:23.877434 24988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "29ac9a39a9804081987566226769aa50"
format_stamp: "Formatted at 2026-08-08 10:07:23 on dist-test-slave-nfb5"
server_key: "4d4d8064b18b87a842d9fb1b85ea0da4"
server_key_iv: "bff24a1928678b5803263062ba9a6de4"
server_key_version: "encryptionkey@0"
I20260808 10:07:23.877518 24988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:23.883782 24978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } attempt: 2
I20260808 10:07:23.884198 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.884059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47472)
0808 10:07:23.884121 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.884161 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:07:23.884555 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.884472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47474)
0808 10:07:23.884515 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.884539 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:07:23.884675 24978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:38337: Network error: Client connection negotiation failed: client connection to 127.24.162.188:38337: connect: Connection refused (error 111)
I20260808 10:07:23.892689 24988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:23.894483 24988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:23.894641 24988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:23.898882 24988 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.188:38337
I20260808 10:07:23.898950 25076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.188:38337 every 8 connection(s)
I20260808 10:07:23.899333 24988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
I20260808 10:07:23.900183 25077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:23.901032 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 24988
I20260808 10:07:23.901127 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183366714887-25226-0/minicluster-data/master-2/wal/instance
I20260808 10:07:23.901307 25226 external_mini_cluster.cc:1428] Setting key 6767aa4e9ba1ad8268f3d131afc0278e
I20260808 10:07:23.902199 25077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:23.912139 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.903191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.24.162.190:45529)
0808 10:07:23.903243 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.903245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.903415 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:07:23.903503 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.903504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.903517 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.903582 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.904099 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.904812 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.906068 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.906348 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.906411 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.908924 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.908927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.908928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.908937 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.910608 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.911099 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.911100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.911100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.911148 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.911435 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.911436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.911437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.911573 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.911640 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.911974 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.912040 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:07:23.912338 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.903093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45529 (local address 127.0.0.1:42356)
0808 10:07:23.903296 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.903340 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.903421 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.903602 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.903604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.903917 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.904010 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.904014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.904945 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.904946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.905995 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.905998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.906437 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.906438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.906507 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.907089 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.907099 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.908812 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.910695 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.910700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.910702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.911024 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.911210 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.911211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.911212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.911359 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.911642 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.911646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.911805 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.911937 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.912090 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260808 10:07:23.912490 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.901862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.24.162.188:38337)
0808 10:07:23.902051 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.902060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.902094 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:23.902495 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.902500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.902544 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.902691 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.903530 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.904852 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.905902 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.906298 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.906418 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.909398 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.909407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.909407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.909415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.911249 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.911643 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.911646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.911646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.911705 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.911895 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.911896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.911896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.912046 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.912175 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.912333 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.912424 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260808 10:07:23.913050 25077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.915182 24881 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } attempt: 2
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:23.921298 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.913383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.24.162.189:34395)
0808 10:07:23.913429 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.913431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.913487 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:07:23.913561 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.913562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.913572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.913627 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.913991 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.914761 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.915692 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.916005 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.916040 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.918186 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.918187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.918187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.918195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.920011 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.920414 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.920415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.920416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.920480 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.920692 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.920693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.920693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.920846 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.920911 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.921112 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.921213 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:23.921736 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.913302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:34395 (local address 127.0.0.1:40852)
0808 10:07:23.913450 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.913460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.913499 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.913735 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.913736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.913868 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.913910 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.913911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.914858 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.914859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.915588 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.915591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.916103 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.916103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.916192 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.916757 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.916762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.918098 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.920108 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.920109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.920110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.920320 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.920539 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.920540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.920541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.920610 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.920941 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.920942 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.920989 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.921512 (+ 523us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.921611 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 10:07:23.922259 25077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:23.922874 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.913678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.24.162.190:45529)
0808 10:07:23.913722 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.913724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.914077 (+ 353us) server_negotiation.cc:407] Connection header received
0808 10:07:23.914125 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.914126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.914133 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.914173 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.914737 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.915264 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.916164 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.916428 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.916515 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.919533 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.919534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.919535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.919543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.921273 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.921705 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.921706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.921707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.921760 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.922148 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.922149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.922149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.922280 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.922501 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.922699 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.922775 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:23.923666 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.915410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47494)
0808 10:07:23.915452 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.915460 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.915491 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.915647 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.915648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.915798 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.915841 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.915843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.916623 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.916624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.917437 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.917440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.917999 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.918001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.918062 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.918460 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.918465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.920210 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.922445 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.922447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.922447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.922695 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.922954 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.922955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.922956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.923030 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.923369 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.923370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.923416 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.923482 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.923579 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 10:07:23.924158 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.915449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.24.162.188:38337)
0808 10:07:23.915481 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.915483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.915490 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:07:23.915537 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.915537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.915575 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.915631 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.915916 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.916562 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.917533 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.917874 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.917973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.920306 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.920307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.920307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.920314 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.922349 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.922786 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.922786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.922787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.922846 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.923115 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.923116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.923116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.923254 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.923338 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.924002 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.924083 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 10:07:23.926074 24881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: Bootstrap starting.
I20260808 10:07:23.926949 24881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:23.927318 24881 log.cc:824] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:23.928267 24881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: No bootstrap required, opened a new log
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:23.930394 24881 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.930541 24881 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3567fc915184a55a55ca87ed81363ff, State: Initialized, Role: FOLLOWER
I20260808 10:07:23.930671 24881 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.930994 25086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.931064 24881 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:23.931176 25086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:23.932199 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.922563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47506)
0808 10:07:23.922948 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.922958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.923000 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.923147 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.923148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.923304 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.923335 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.923336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.924935 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.924936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.925676 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.925678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.926172 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.926173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.926227 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.926666 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.926671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.928712 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.930858 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.930859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.930860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.931100 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.931292 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.931293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.931293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.931356 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.931752 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.931753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.931798 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.931886 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.932015 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344}
I20260808 10:07:23.932327 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.922630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.24.162.188:38337)
0808 10:07:23.922999 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.923001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.923007 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:07:23.923055 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.923056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.923072 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.923141 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.923423 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.924848 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.925772 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.926079 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.926143 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.928813 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.928815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.928815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.928823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.930764 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.931175 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.931176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.931177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.931226 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.931458 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.931459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.931459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.931648 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.931724 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.931939 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.932087 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":346,"thread_start_us":119,"threads_started":1}
I20260808 10:07:23.932693 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.923763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.24.162.189:34395)
0808 10:07:23.923882 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.923884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.923891 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:07:23.923910 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.923910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.923918 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.923970 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.924572 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.925284 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.926113 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.926401 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.926445 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.928880 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.928881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.928882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.928889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.931033 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.931447 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.931448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.931448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.931502 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.931714 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.931715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.931715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.931854 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.932458 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.932504 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.932594 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 10:07:23.933921 25077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: Bootstrap starting.
W20260808 10:07:23.934118 25098 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:23.934162 25098 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:23.934228 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:23.934688 25077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:23.935091 25077 log.cc:824] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:23.935123 24844 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a3567fc915184a55a55ca87ed81363ff"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42364
I20260808 10:07:23.935231 24844 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:07:23.935271 24844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:23.936533 24844 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.936786 24844 leader_election.cc:288] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [CANDIDATE]: Term 1 election: Requested vote from peers 7998c663048c4fc9a34c6d64d0a9908c (127.24.162.189:34395), 29ac9a39a9804081987566226769aa50 (127.24.162.188:38337)
I20260808 10:07:23.935091 24788 master_runner.cc:186] Error getting master registration for 127.24.162.189:34395: OK, instance_id { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" instance_seqno: 1786183643740210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:07:23.937650 24788 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:07:23.947696 25077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: No bootstrap required, opened a new log
I20260808 10:07:23.949163 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.938669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.24.162.190:45529)
0808 10:07:23.938723 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.938725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.939115 (+ 390us) server_negotiation.cc:407] Connection header received
0808 10:07:23.939294 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.939295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.939307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.939348 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.940517 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.941275 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.942656 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.942987 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.943033 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.944774 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.944775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.944776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.944784 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.946624 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.947663 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.947664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.947664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.947749 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.948209 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.948210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.948211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.948366 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.948471 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.948966 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.949049 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:23.949234 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.938861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.24.162.189:34395)
0808 10:07:23.938906 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.938908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.939247 (+ 339us) server_negotiation.cc:407] Connection header received
0808 10:07:23.939296 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.939297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.939307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.939348 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.940135 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.940881 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.942020 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.942348 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.942417 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.944460 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.944461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.944462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.944482 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.947527 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.948234 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.948236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.948236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.948295 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.948529 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.948530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.948531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.948671 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.948736 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.948990 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.949073 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:07:23.949571 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.939108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.24.162.188:38337)
0808 10:07:23.939162 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.939164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.940229 (+ 1065us) server_negotiation.cc:407] Connection header received
0808 10:07:23.940338 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.940339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.940352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.940384 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.941089 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.941826 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.942859 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.943231 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.943303 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.946368 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.946370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.946370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.946377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.948272 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.948697 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.948699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.948699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.948752 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.948961 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.948962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.948963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.949087 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.949170 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.949400 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.949481 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:07:23.950147 25077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.950299 25077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ac9a39a9804081987566226769aa50, State: Initialized, Role: FOLLOWER
I20260808 10:07:23.950382 25077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.950697 25077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:23.950760 25046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3567fc915184a55a55ca87ed81363ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29ac9a39a9804081987566226769aa50"
I20260808 10:07:23.950848 25046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:23.952827 25046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3567fc915184a55a55ca87ed81363ff in term 1.
I20260808 10:07:23.953018 25112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.953091 24811 leader_election.cc:302] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ac9a39a9804081987566226769aa50, a3567fc915184a55a55ca87ed81363ff; no voters:
I20260808 10:07:23.953235 25086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:23.953418 25086 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [term 1 LEADER]: Becoming Leader. State: Replica: a3567fc915184a55a55ca87ed81363ff, State: Running, Role: LEADER
I20260808 10:07:23.953506 25086 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.954083 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3567fc915184a55a55ca87ed81363ff. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.954185 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:23.953125 25112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:07:23.953835 25128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:23.954288 25128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:23.955847 24988 master_runner.cc:186] Error getting master registration for 127.24.162.189:34395: OK, instance_id { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" instance_seqno: 1786183643740210 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:07:23.958084 24988 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:07:23.958243 25134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:23.961249 25134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:23.962946 25046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [term 1 FOLLOWER]: Refusing update from remote peer a3567fc915184a55a55ca87ed81363ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:23.963280 25130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:23.964555 25112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3567fc915184a55a55ca87ed81363ff. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.964685 25112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:23.966176 25086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.966284 25086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:23.966404 25112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.966490 25112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:23.966836 25136 mvcc.cc:205] Tried to move back new op lower bound from 7316208205669560320 to 7316208205634498560. Current Snapshot: MvccSnapshot[applied={T|T < 7316208205669560320}]
I20260808 10:07:23.967307 25134 catalog_manager.cc:1347] Generated new cluster ID: 21b726d9f1f0496c80ab126e51c5eb8f
I20260808 10:07:23.967372 25134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:23.975823 24978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } attempt: 3
Aug 08 10:07:23 dist-test-slave-nfb5 krb5kdc[24770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183643, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:23.982064 25134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:23.984771 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.976138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:38337 (local address 127.0.0.1:47512)
0808 10:07:23.976225 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:23.976237 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:23.976276 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:23.976522 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:23.976523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:23.976722 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:23.976771 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.976773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.977679 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.977680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.978494 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:23.978496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.978984 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.978984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:23.979055 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.979653 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:23.979659 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:23.981705 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:23.983713 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.983714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.983715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.983926 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.984123 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:23.984124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:23.984124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:23.984181 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:23.984464 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:23.984465 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:23.984509 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:07:23.984570 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:07:23.984667 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 10:07:23.985109 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:23.976346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.24.162.188:38337)
0808 10:07:23.976393 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:23.976395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:23.976406 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:23.976427 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:23.976428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:23.976438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:23.976504 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:23.976844 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.977610 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.978568 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:23.978881 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:23.978976 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:23.981798 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:23.981800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:23.981800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:23.981808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:23.983617 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.984003 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.984004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.984005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.984058 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:23.984242 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:23.984243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:23.984243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:23.984378 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:23.984437 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:23.984942 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 10:07:23.985022 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:07:23.985739 25134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:23.986742 24978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: Bootstrap starting.
I20260808 10:07:23.987495 24978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:23.987809 24978 log.cc:824] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:23.988976 24978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: No bootstrap required, opened a new log
I20260808 10:07:23.991190 24978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.991321 24978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7998c663048c4fc9a34c6d64d0a9908c, State: Initialized, Role: FOLLOWER
I20260808 10:07:23.991394 24978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } }
I20260808 10:07:23.991719 24978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:23.992630 24947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:23.993495 24947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [term 1 FOLLOWER]: Refusing update from remote peer a3567fc915184a55a55ca87ed81363ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:23.993688 24948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3567fc915184a55a55ca87ed81363ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7998c663048c4fc9a34c6d64d0a9908c"
I20260808 10:07:23.994421 24948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3567fc915184a55a55ca87ed81363ff in term 1.
I20260808 10:07:23.994953 25086 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:23.996042 25134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff: Generated new TSK 0
I20260808 10:07:23.996217 25134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:23.997243 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:23.997344 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3567fc915184a55a55ca87ed81363ff [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:23.998634 25165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:23.998692 25165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:24.000607 25148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:24.000852 25148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:24.000949 25148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3567fc915184a55a55ca87ed81363ff. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:24.001046 25148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:24.001116 25148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3567fc915184a55a55ca87ed81363ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3567fc915184a55a55ca87ed81363ff" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45529 } } peers { permanent_uuid: "7998c663048c4fc9a34c6d64d0a9908c" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 34395 } } peers { permanent_uuid: "29ac9a39a9804081987566226769aa50" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 38337 } } }
I20260808 10:07:24.001204 25148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:24.001919 25145 mvcc.cc:205] Tried to move back new op lower bound from 7316208205794963456 to 7316208205634498560. Current Snapshot: MvccSnapshot[applied={T|T < 7316208205669560320}]
W20260808 10:07:24.031749 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.029969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.24.162.190:45529)
0808 10:07:24.030017 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.030019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.031033 (+ 1014us) server_negotiation.cc:407] Connection header received
0808 10:07:24.031156 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.031157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.031170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.031205 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.031670 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42386: BlockingRecv error: recv got EOF from 127.0.0.1:42386 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 10:07:24.032612 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.030352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.24.162.189:34395)
0808 10:07:24.030393 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.030395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.031998 (+ 1603us) server_negotiation.cc:407] Connection header received
0808 10:07:24.032132 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.032133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.032144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.032174 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.032553 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40884: BlockingRecv error: recv got EOF from 127.0.0.1:40884 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 10:07:24.033388 25084 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.030651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.24.162.188:38337)
0808 10:07:24.030693 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.030694 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.032870 (+ 2176us) server_negotiation.cc:407] Connection header received
0808 10:07:24.032969 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.032970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.032981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.033011 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.033313 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47524: BlockingRecv error: recv got EOF from 127.0.0.1:47524 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 10:07:24.036018 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.035100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.24.162.189:34395)
0808 10:07:24.035134 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.035136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.035487 (+ 351us) server_negotiation.cc:407] Connection header received
0808 10:07:24.035591 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.035592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.035609 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.035639 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.035960 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40886: BlockingRecv error: recv got EOF from 127.0.0.1:40886 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 10:07:24.039004 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.038210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.24.162.189:34395)
0808 10:07:24.038242 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.038244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.038491 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:07:24.038581 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.038582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.038591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.038619 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.038964 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40898: BlockingRecv error: recv got EOF from 127.0.0.1:40898 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 10:07:24.042748 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.042022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.24.162.189:34395)
0808 10:07:24.042055 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.042057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.042281 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:07:24.042371 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.042372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.042381 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.042409 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.042707 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40914: BlockingRecv error: recv got EOF from 127.0.0.1:40914 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 10:07:24.048620 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.047816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.24.162.189:34395)
0808 10:07:24.047860 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.047861 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.048139 (+ 278us) server_negotiation.cc:407] Connection header received
0808 10:07:24.048232 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.048232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.048242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.048270 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.048577 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40924: BlockingRecv error: recv got EOF from 127.0.0.1:40924 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 10:07:24.058444 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.057650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.24.162.189:34395)
0808 10:07:24.057685 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.057687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.057942 (+ 255us) server_negotiation.cc:407] Connection header received
0808 10:07:24.058036 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.058037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.058047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.058075 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.058397 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40928: BlockingRecv error: recv got EOF from 127.0.0.1:40928 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 10:07:24.076604 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.075655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.24.162.189:34395)
0808 10:07:24.075695 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.075697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.075994 (+ 297us) server_negotiation.cc:407] Connection header received
0808 10:07:24.076100 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.076101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.076112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.076142 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.076544 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40930: BlockingRecv error: recv got EOF from 127.0.0.1:40930 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 10:07:24.111128 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.109999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.24.162.189:34395)
0808 10:07:24.110073 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.110076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.110438 (+ 362us) server_negotiation.cc:407] Connection header received
0808 10:07:24.110550 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.110551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.110563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.110597 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.111061 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40940: BlockingRecv error: recv got EOF from 127.0.0.1:40940 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 10:07:24.177129 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.176477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.24.162.189:34395)
0808 10:07:24.176546 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.176548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.176617 (+ 69us) server_negotiation.cc:407] Connection header received
0808 10:07:24.176679 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.176680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.176693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.176738 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.177078 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40956: BlockingRecv error: recv got EOF from 127.0.0.1:40956 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 10:07:24.307047 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.306310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.24.162.189:34395)
0808 10:07:24.306411 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.306413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.306425 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:24.306502 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.306503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.306514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.306568 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.306963 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40958: BlockingRecv error: recv got EOF from 127.0.0.1:40958 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 10:07:24.564869 24985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:24.564264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.24.162.189:34395)
0808 10:07:24.564331 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:24.564333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:24.564343 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:24.564388 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:24.564389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:24.564399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:24.564441 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:24.564789 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40962: BlockingRecv error: recv got EOF from 127.0.0.1:40962 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:24.954723 25128 catalog_manager.cc:1259] Loaded cluster ID: 21b726d9f1f0496c80ab126e51c5eb8f
I20260808 10:07:24.954799 25128 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: loading cluster ID for follower catalog manager: success
I20260808 10:07:24.955664 25128 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: acquiring CA information for follower catalog manager: success
I20260808 10:07:24.956010 25128 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29ac9a39a9804081987566226769aa50: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:07:24.999188 25165 catalog_manager.cc:1259] Loaded cluster ID: 21b726d9f1f0496c80ab126e51c5eb8f
I20260808 10:07:24.999274 25165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: loading cluster ID for follower catalog manager: success
I20260808 10:07:25.000103 25165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: acquiring CA information for follower catalog manager: success
I20260808 10:07:25.000384 25165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7998c663048c4fc9a34c6d64d0a9908c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:07:25.080672 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:25.077996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.24.162.189:34395)
0808 10:07:25.078128 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:25.078130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:25.078141 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:25.078195 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:25.078195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:25.078248 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:25.078321 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:25.078460 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.079024 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.079777 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.079971 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.080023 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:25.080178 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:25.080307 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:25.080369 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:25.080485 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 10:07:25.080504 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260808 10:07:25.085112 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:25.082588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.24.162.188:38337)
0808 10:07:25.082718 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:25.082720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:25.082729 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:25.082774 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:25.082775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:25.082828 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:25.082920 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:25.083066 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.083553 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.084285 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.084504 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.084590 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:25.084755 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:25.084865 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:25.084946 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:25.084999 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 10:07:25.085019 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
I20260808 10:07:25.089973 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:25.086730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.24.162.190:45529)
0808 10:07:25.086845 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:25.086847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:25.086863 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:07:25.086901 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:25.086901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:25.086996 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:25.087098 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:25.087228 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.087941 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.088953 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.089279 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.089374 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:25.089544 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:25.089673 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:25.089779 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:25.089832 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 10:07:25.089861 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260808 10:07:25.090014 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:25.086891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.24.162.189:34395)
0808 10:07:25.086965 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:25.086968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:25.087128 (+ 160us) server_negotiation.cc:407] Connection header received
0808 10:07:25.087254 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:25.087255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:25.087268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:25.087320 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:25.087472 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.088211 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.089173 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.089475 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.089537 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:25.089729 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:25.089762 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:25.089812 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:25.089905 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:07:25.089928 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:07:25.090047 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:25.087223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.24.162.188:38337)
0808 10:07:25.087260 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:25.087262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:25.087469 (+ 207us) server_negotiation.cc:407] Connection header received
0808 10:07:25.087550 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:25.087551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:25.087562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:25.087612 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:25.087709 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.088369 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.089247 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:25.089542 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:25.089606 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:25.089775 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:25.089810 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:25.089840 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:25.089948 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:07:25.089969 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:25.091686 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24788
I20260808 10:07:25.107235 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24890
I20260808 10:07:25.116901 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 24988
2026-08-08T10:07:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3029 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:25 dist-test-slave-nfb5 krb5kdc[25292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:25 dist-test-slave-nfb5 krb5kdc[25292](info): set up 2 sockets
Aug 08 10:07:25 dist-test-slave-nfb5 krb5kdc[25292](info): commencing operation
krb5kdc: starting...
W20260808 10:07:26.351598 25226 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/test-user.keytab.
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.162.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
Entry for principal kudu/127.24.162.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab.
I20260808 10:07:26.475401 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45581
--webserver_interface=127.24.162.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:26.559674 25316 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:26.559888 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:26.559906 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:26.559926 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:26.561431 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:26.561509 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:26.561532 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:26.561550 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:26.561578 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:26.563665 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.190.keytab
--principal=kudu/127.24.162.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.190:45581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.162.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:26.563958 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:26.564258 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:26.569260 25327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:26.569235 25325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.569288 25316 server_base.cc:1034] running on GCE node
W20260808 10:07:26.569243 25335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.569686 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:26.569958 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:26.571096 25316 hybrid_clock.cc:648] HybridClock initialized: now 1786183646571078 us; error 32 us; skew 500 ppm
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:26.573563 25316 init.cc:377] Logged in from keytab as kudu/127.24.162.190@KRBTEST.COM (short username kudu)
I20260808 10:07:26.574009 25316 webserver.cc:460] Webserver started at http://127.24.162.190:45413/ using document root <none> and password file <none>
I20260808 10:07:26.574252 25316 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:26.574322 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:26.574491 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:26.575627 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data/instance:
uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "1781e7d190b96330ed4bdfce3a704bf6"
server_key_iv: "b457502c2d8ac79223f95669aea4290e"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.575992 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance:
uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "1781e7d190b96330ed4bdfce3a704bf6"
server_key_iv: "b457502c2d8ac79223f95669aea4290e"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.577363 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:26.578123 25342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:26.578312 25316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:26.578423 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal
uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "1781e7d190b96330ed4bdfce3a704bf6"
server_key_iv: "b457502c2d8ac79223f95669aea4290e"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.578500 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:26.590687 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:26.593065 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:26.593232 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:26.597447 25316 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.190:45581
I20260808 10:07:26.597486 25415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.190:45581 every 8 connection(s)
I20260808 10:07:26.597903 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:26.598986 25417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:26.600564 25417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.601158 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25316
I20260808 10:07:26.601275 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-0/wal/instance
I20260808 10:07:26.601483 25226 external_mini_cluster.cc:1428] Setting key 3dabcdfbba93491ac761f5e4105a61dc
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.162.189@KRBTEST.COM; defaulting to no policy
I20260808 10:07:26.612454 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.601317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.24.162.190:45581)
0808 10:07:26.601506 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.601516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.601796 (+ 280us) server_negotiation.cc:407] Connection header received
0808 10:07:26.602566 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.602567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.602584 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.602661 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.603521 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.604831 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.606223 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.606566 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.606655 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.609093 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.609100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.609101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.609110 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.610841 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.611236 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.611240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.611240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.611324 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.611563 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.611564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.611564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.611773 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.611874 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.612071 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.612276 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260808 10:07:26.612465 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.601284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45581 (local address 127.0.0.1:54776)
0808 10:07:26.601627 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.601675 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.602415 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.602719 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.602722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.603149 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.603401 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.603408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.604937 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.604938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.606130 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.606134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.606665 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.606666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.606764 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.607341 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.607352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.609000 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.610937 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.610944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.610946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.611140 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.611401 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.611402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.611402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.611492 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.611853 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.611859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.611920 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.612023 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.612207 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.189@KRBTEST.COM" created.
I20260808 10:07:26.613359 25417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.613766 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.613624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49528)
0808 10:07:26.613677 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.613731 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:07:26.613898 25345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
I20260808 10:07:26.614326 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.614230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49534)
0808 10:07:26.614278 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.614303 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:07:26.614984 25417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:45927: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
Entry for principal kudu/127.24.162.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab.
I20260808 10:07:26.628011 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:45927
--webserver_interface=127.24.162.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:07:26.664225 25417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } attempt: 1
I20260808 10:07:26.664712 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.664569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49544)
0808 10:07:26.664641 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.664679 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:07:26.665086 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.664989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49552)
0808 10:07:26.665034 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.665061 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:07:26.665176 25417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:45927: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
W20260808 10:07:26.710350 25425 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:26.710625 25425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:26.710644 25425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:26.710702 25425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:26.711872 25425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:26.711916 25425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:26.711930 25425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:26.711974 25425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:26.712005 25425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:26.713450 25425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.189.keytab
--principal=kudu/127.24.162.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.189:45927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.162.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:26.713732 25425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:26.713971 25425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:26.719036 25432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:26.719048 25433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.719249 25425 server_base.cc:1034] running on GCE node
W20260808 10:07:26.719251 25435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.719511 25425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:26.719831 25425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:26.721017 25425 hybrid_clock.cc:648] HybridClock initialized: now 1786183646721003 us; error 48 us; skew 500 ppm
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:26.723624 25425 init.cc:377] Logged in from keytab as kudu/127.24.162.189@KRBTEST.COM (short username kudu)
I20260808 10:07:26.724157 25425 webserver.cc:460] Webserver started at http://127.24.162.189:35659/ using document root <none> and password file <none>
I20260808 10:07:26.724403 25425 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:26.724472 25425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:26.724632 25425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:26.725736 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data/instance:
uuid: "72aacf4f312b4a9b880f42f84e92fe6a"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "95979bb3c10f808ca7c0e6c3986163cb"
server_key_iv: "13d7f1d04e467c3709c4959e792c304f"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.726116 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal/instance:
uuid: "72aacf4f312b4a9b880f42f84e92fe6a"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "95979bb3c10f808ca7c0e6c3986163cb"
server_key_iv: "13d7f1d04e467c3709c4959e792c304f"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.727588 25425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:26.728384 25444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:26.728569 25425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:26.728677 25425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal
uuid: "72aacf4f312b4a9b880f42f84e92fe6a"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "95979bb3c10f808ca7c0e6c3986163cb"
server_key_iv: "13d7f1d04e467c3709c4959e792c304f"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.728771 25425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:26.733330 25417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } attempt: 2
I20260808 10:07:26.733753 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.733625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49568)
0808 10:07:26.733686 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.733724 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:07:26.734114 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.734021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49582)
0808 10:07:26.734066 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.734090 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:07:26.734200 25417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.189:45927: Network error: Client connection negotiation failed: client connection to 127.24.162.189:45927: connect: Connection refused (error 111)
I20260808 10:07:26.751103 25425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:26.753153 25425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:26.753331 25425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:26.757740 25425 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.189:45927
I20260808 10:07:26.757783 25523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.189:45927 every 8 connection(s)
I20260808 10:07:26.758181 25425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/data/info.pb
I20260808 10:07:26.759085 25524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:26.760656 25524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.763248 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25425
I20260808 10:07:26.763365 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-1/wal/instance
I20260808 10:07:26.763569 25226 external_mini_cluster.cc:1428] Setting key bfbdb199eb25aaa68deacce9b24b49e1
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:26.770598 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.761296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45581 (local address 127.0.0.1:54786)
0808 10:07:26.761487 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.761525 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.761982 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.762187 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.762190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.762447 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.762636 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.762640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.763389 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.763391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.764660 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.764664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.765040 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.765041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.765193 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.765895 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.765903 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.767446 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.769136 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.769145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.769147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.769418 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.769663 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.769664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.769664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.769767 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.770087 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.770092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.770207 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.770291 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.770430 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260808 10:07:26.770592 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.761353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54786 (local address 127.24.162.190:45581)
0808 10:07:26.761446 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.761448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.761612 (+ 164us) server_negotiation.cc:407] Connection header received
0808 10:07:26.762068 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.762069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.762082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.762136 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.762711 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.763264 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.764757 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.764949 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.765009 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.767556 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.767558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.767558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.767565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.769048 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.769510 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.769512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.769512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.769586 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.769838 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.769839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.769839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.770012 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.770095 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.770353 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.770455 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:07:26.771387 25524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.162.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.162.188@KRBTEST.COM" created.
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:26.780169 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.771740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49596 (local address 127.24.162.189:45927)
0808 10:07:26.771909 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.771913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.771927 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:07:26.771946 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.771948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.771958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.772021 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.772354 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.773015 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.773926 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.774225 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.774281 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.777045 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.777053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.777053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.777060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.779045 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.779372 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.779375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.779375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.779445 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.779647 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.779647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.779647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.779784 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.779846 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.780014 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.780086 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260808 10:07:26.780323 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.771700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49596)
0808 10:07:26.771761 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.771775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.771824 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.772044 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.772045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.772205 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.772266 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.772267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.773105 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.773106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.773853 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.773856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.774307 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.774308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.774383 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.774895 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.774901 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.776934 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.779125 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.779126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.779127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.779285 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.779514 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.779516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.779516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.779584 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.779860 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.779861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.779911 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.779991 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.780154 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":54}
I20260808 10:07:26.781023 25524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.781410 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.781287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58872)
0808 10:07:26.781338 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.781384 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:07:26.781540 25445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
I20260808 10:07:26.781991 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.781896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58888)
0808 10:07:26.781942 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.781967 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:07:26.782104 25524 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:39751: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
Entry for principal kudu/127.24.162.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab.
I20260808 10:07:26.793629 25226 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHQacpx/build/release/bin/kudu
/tmp/dist-test-taskHQacpx/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:39751
--webserver_interface=127.24.162.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.162.148:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:07:26.825362 25417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } attempt: 3
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:26.831321 25524 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } attempt: 1
I20260808 10:07:26.831760 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.831630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58902)
0808 10:07:26.831686 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.831717 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:07:26.832233 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.832122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58910)
0808 10:07:26.832168 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.832200 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:07:26.832401 25524 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:39751: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
I20260808 10:07:26.833464 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.825794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49598)
0808 10:07:26.825862 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.825875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.825955 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.826108 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.826109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.826304 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.826372 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.826374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.827130 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.827132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.827671 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.827673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.828080 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.828081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.828161 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.828681 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.828687 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.830138 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.832504 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.832505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.832505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.832685 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.832908 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.832908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.832908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.832952 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.833225 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.833226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.833268 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.833321 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.833401 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:07:26.833537 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.825877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.24.162.189:45927)
0808 10:07:26.825951 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.825953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.825966 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:07:26.826019 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.826022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.826038 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.826103 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.826468 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.827045 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.827763 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.827992 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.828071 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.830237 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.830238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.830238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.830245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.832366 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.832762 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.832764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.832764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.832841 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.833006 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.833008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.833008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.833146 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.833217 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.833372 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.833456 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:07:26.834124 25417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.834465 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.834359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58922)
0808 10:07:26.834406 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.834432 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 10:07:26.834824 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.834762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58938)
0808 10:07:26.834787 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.834809 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 10:07:26.834941 25417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:39751: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
I20260808 10:07:26.866091 25417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } attempt: 1
I20260808 10:07:26.866546 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.866413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58950)
0808 10:07:26.866480 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.866517 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:07:26.866997 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.866858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58966)
0808 10:07:26.866907 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.866972 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:07:26.867098 25417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:39751: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
W20260808 10:07:26.878094 25532 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:07:26.878332 25532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:26.878387 25532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:26.878432 25532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:26.879643 25532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:26.879705 25532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:26.879731 25532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:26.879772 25532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:26.879802 25532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:26.881374 25532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.162.148:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.162.190:45581,127.24.162.189:45927,127.24.162.188:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/krb5kdc/kudu_127.24.162.188.keytab
--principal=kudu/127.24.162.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.162.188:39751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.162.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:26.881671 25532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:26.881929 25532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:26.886806 25544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:26.886793 25539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.886885 25532 server_base.cc:1034] running on GCE node
W20260808 10:07:26.886791 25542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:26.887358 25532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:26.887682 25532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:26.888855 25532 hybrid_clock.cc:648] HybridClock initialized: now 1786183646888847 us; error 36 us; skew 500 ppm
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:26.891449 25532 init.cc:377] Logged in from keytab as kudu/127.24.162.188@KRBTEST.COM (short username kudu)
I20260808 10:07:26.891884 25532 webserver.cc:460] Webserver started at http://127.24.162.188:43501/ using document root <none> and password file <none>
I20260808 10:07:26.892109 25532 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:26.892186 25532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:26.892345 25532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:26.893425 25532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data/instance:
uuid: "8be5133e86d0494cadbe3c4dce6c54a8"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "208f6e5b5d77176edc44381db22dd979"
server_key_iv: "6a8c5557e5641dc4a6e4bf8e938539f4"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.893797 25532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal/instance:
uuid: "8be5133e86d0494cadbe3c4dce6c54a8"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "208f6e5b5d77176edc44381db22dd979"
server_key_iv: "6a8c5557e5641dc4a6e4bf8e938539f4"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.895185 25532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:26.895893 25554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:26.896064 25532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:26.896183 25532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data,/tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal
uuid: "8be5133e86d0494cadbe3c4dce6c54a8"
format_stamp: "Formatted at 2026-08-08 10:07:26 on dist-test-slave-nfb5"
server_key: "208f6e5b5d77176edc44381db22dd979"
server_key_iv: "6a8c5557e5641dc4a6e4bf8e938539f4"
server_key_version: "encryptionkey@0"
I20260808 10:07:26.896258 25532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:26.900549 25524 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } attempt: 2
I20260808 10:07:26.900934 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.900808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58982)
0808 10:07:26.900867 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.900904 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:07:26.901280 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.901201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:58998)
0808 10:07:26.901238 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.901262 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 10:07:26.901427 25524 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.162.188:39751: Network error: Client connection negotiation failed: client connection to 127.24.162.188:39751: connect: Connection refused (error 111)
I20260808 10:07:26.921906 25532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:26.923918 25532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:26.924077 25532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:26.928040 25532 rpc_server.cc:307] RPC server started. Bound to: 127.24.162.188:39751
I20260808 10:07:26.928092 25627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.162.188:39751 every 8 connection(s)
I20260808 10:07:26.928471 25532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/data/info.pb
I20260808 10:07:26.929412 25628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:26.929744 25226 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHQacpx/build/release/bin/kudu as pid 25532
I20260808 10:07:26.929875 25226 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHQacpx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183366714887-25226-0/minicluster-data/master-2/wal/instance
I20260808 10:07:26.930074 25226 external_mini_cluster.cc:1428] Setting key 0aa54471775d3d44f66e12379807f353
I20260808 10:07:26.931597 25628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:26.940601 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.932308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.24.162.190:45581)
0808 10:07:26.932376 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.932378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.932579 (+ 201us) server_negotiation.cc:407] Connection header received
0808 10:07:26.932619 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.932621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.932634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.932697 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.933255 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.933698 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.934614 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.934815 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.934871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.937230 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.937232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.937232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.937238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.939096 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.939516 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.939517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.939518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.939582 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.939891 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.939892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.939892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.940000 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.940094 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.940401 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.940484 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:07:26.940886 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.932265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.190:45581 (local address 127.0.0.1:54798)
0808 10:07:26.932446 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.932480 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.932568 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.932707 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.932709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.933054 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.933166 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.933172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.933777 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.933778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.934528 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.934530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.934896 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.934897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.934982 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.935636 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.935645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.937116 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.939171 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.939178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.939180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.939439 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.939656 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.939657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.939657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.939806 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.940091 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.940097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.940252 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.940367 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.940491 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260808 10:07:26.941032 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.930727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.24.162.188:39751)
0808 10:07:26.930966 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.930976 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.931009 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:07:26.931543 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.931551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.931601 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.931758 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.932676 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.933777 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.934483 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.934836 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.934979 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.937770 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.937779 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.937780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.937788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.939675 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.940059 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.940063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.940063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.940132 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.940327 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.940328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.940328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.940512 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.940668 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.940792 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.940869 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260808 10:07:26.941574 25628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.942224 25417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } attempt: 2
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.190@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.190@KRBTEST.COM
I20260808 10:07:26.949890 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.942476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:59008)
0808 10:07:26.942525 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.942535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.942578 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.942701 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.942702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.942856 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.942891 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.942892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.943556 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.943557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.944294 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.944297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.944790 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.944852 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.945273 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.945278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.946811 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.948685 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.948686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.948687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.948971 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.949194 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.949195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.949262 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.949551 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.949552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.949597 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.949650 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.949768 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260808 10:07:26.950412 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.941883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.189:45927 (local address 127.0.0.1:49612)
0808 10:07:26.942005 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.942017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.942087 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.942221 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.942222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.942387 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.942445 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.942447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.943411 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.943411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.944047 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.944049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.944429 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.944495 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.944948 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.944953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.947184 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.949280 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.949281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.949498 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.949722 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.949723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.949787 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.950077 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.950078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.950120 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.950195 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.950299 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 10:07:26.950501 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.941945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.24.162.189:45927)
0808 10:07:26.942008 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.942010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.942063 (+ 53us) server_negotiation.cc:407] Connection header received
0808 10:07:26.942153 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.942155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.942165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.942221 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.942546 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.943311 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944138 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.944357 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.947268 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.947270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.947270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.947278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.949198 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949577 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.949578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.949579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.949641 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949863 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.949864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.949864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.950015 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.950087 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.950322 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.950402 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:07:26.950870 25628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:07:26.950809 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.942508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.24.162.188:39751)
0808 10:07:26.942541 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.942542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.942565 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:07:26.942625 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.942625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.942634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.942684 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.942987 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.943469 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944386 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.944703 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944776 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.946899 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.946901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.946901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.946908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.948584 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949057 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.949058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.949059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.949119 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949338 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.949339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.949339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.949462 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.950230 (+ 768us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.950268 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.950350 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":15}
I20260808 10:07:26.951033 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.942074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.24.162.190:45581)
0808 10:07:26.942109 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.942110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.942117 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:07:26.942139 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.942140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.942149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.942372 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.943075 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.943735 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.944644 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.944943 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.945011 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.947864 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.947865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.947866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.947873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.949617 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.949994 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.949995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.949996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.950050 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.950311 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.950311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.950312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.950434 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.950593 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.950834 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.950905 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:07:26.952880 25417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Bootstrap starting.
I20260808 10:07:26.953662 25417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:26.953984 25417 log.cc:824] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:26.954843 25417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: No bootstrap required, opened a new log
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.188@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:26.956727 25417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:26.956885 25417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bd7d4bdd12e41ee967c5cbfbdb2d8ab, State: Initialized, Role: FOLLOWER
I20260808 10:07:26.956971 25417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.162.189@KRBTEST.COM
I20260808 10:07:26.957316 25417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:26.957273 25634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.957404 25634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:26.958421 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.951119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:59018)
0808 10:07:26.951356 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.951366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.951404 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.951634 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.951635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.951779 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.951808 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.951809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.952653 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.952653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.953369 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.953373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.953827 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.953828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.953868 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.954284 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.954289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.955797 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.957258 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.957258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.957259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.957476 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.957669 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.957670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.957670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.957723 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.958062 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:26.958063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:26.958101 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:07:26.958188 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:07:26.958283 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260808 10:07:26.958429 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.951458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.24.162.188:39751)
0808 10:07:26.951516 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.951517 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.951523 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:07:26.951560 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.951561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.951570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.951624 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.951888 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.952563 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.953460 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.953725 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.953787 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.955892 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.955893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.955894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.955900 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.957172 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.957538 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.957539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.957539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.957590 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.957794 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.957794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.957795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.957975 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.958066 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.958208 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.958333 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":37}
I20260808 10:07:26.959900 25628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: Bootstrap starting.
I20260808 10:07:26.960958 25628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:26.961127 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.951827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.24.162.189:45927)
0808 10:07:26.951862 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.951864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.951872 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:26.951931 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.951932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.951941 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.951997 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.952451 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.952880 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.953602 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.953911 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.953988 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.957931 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.957933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.957934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.957942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.959845 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.960237 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.960238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.960238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.960311 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.960555 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.960556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.960556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.960706 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.960883 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.960944 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.961021 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:26.961262 25628 log.cc:824] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: Log is configured to *not* fsync() on all Append() calls
W20260808 10:07:26.959614 25653 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:26.961753 25653 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:26.962194 25628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: No bootstrap required, opened a new log
I20260808 10:07:26.962667 25226 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:26.963683 25380 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54804
I20260808 10:07:26.963786 25380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:07:26.963840 25380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:26.964367 25628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:26.964501 25628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be5133e86d0494cadbe3c4dce6c54a8, State: Initialized, Role: FOLLOWER
I20260808 10:07:26.964574 25628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:26.964687 25380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:26.964879 25628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:26.964922 25380 leader_election.cc:288] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [CANDIDATE]: Term 1 election: Requested vote from peers 72aacf4f312b4a9b880f42f84e92fe6a (127.24.162.189:45927), 8be5133e86d0494cadbe3c4dce6c54a8 (127.24.162.188:39751)
I20260808 10:07:26.965243 25590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8be5133e86d0494cadbe3c4dce6c54a8"
I20260808 10:07:26.965345 25590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:26.962683 25316 master_runner.cc:186] Error getting master registration for 127.24.162.189:45927: OK, instance_id { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" instance_seqno: 1786183646756526 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:07:26.965699 25316 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:07:26.966213 25590 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bd7d4bdd12e41ee967c5cbfbdb2d8ab in term 1.
I20260808 10:07:26.966394 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.966444 25343 leader_election.cc:302] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bd7d4bdd12e41ee967c5cbfbdb2d8ab, 8be5133e86d0494cadbe3c4dce6c54a8; no voters:
I20260808 10:07:26.966481 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:26.966562 25634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:26.966709 25634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [term 1 LEADER]: Becoming Leader. State: Replica: 8bd7d4bdd12e41ee967c5cbfbdb2d8ab, State: Running, Role: LEADER
I20260808 10:07:26.966801 25634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:26.967278 25634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bd7d4bdd12e41ee967c5cbfbdb2d8ab. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.967389 25634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:26.969055 25681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:26.969269 25681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 10:07:26.970954 25671 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:26.971015 25671 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:26.969008 25532 master_runner.cc:186] Error getting master registration for 127.24.162.189:45927: OK, instance_id { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" instance_seqno: 1786183646756526 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:07:26.971338 25532 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:07:26.976720 25590 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [term 1 FOLLOWER]: Refusing update from remote peer 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:26.977061 25634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:26.977507 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.967621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.24.162.190:45581)
0808 10:07:26.967799 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.967801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.968611 (+ 810us) server_negotiation.cc:407] Connection header received
0808 10:07:26.968696 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.968697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.968708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.968760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.969848 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.970550 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.971810 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.972045 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.972110 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.974302 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.974304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.974304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.974311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.976143 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.976574 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.976575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.976575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.976637 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.976893 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.976894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.976894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.977013 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.977099 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.977327 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.977409 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260808 10:07:26.978389 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bd7d4bdd12e41ee967c5cbfbdb2d8ab. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.978488 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:26.979373 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.967836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.24.162.189:45927)
0808 10:07:26.967874 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.967875 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.971189 (+ 3314us) server_negotiation.cc:407] Connection header received
0808 10:07:26.971284 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.971285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.971295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.971324 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.972044 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.972732 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.973772 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.974079 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.974121 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.975616 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.975617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.975617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.975625 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.977445 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.978056 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.978058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.978058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.978118 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.978435 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.978437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.978437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.978596 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.978704 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.979174 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.979259 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:26.983269 25634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.983359 25634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:26.983500 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:26.983584 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:26.983598 25688 mvcc.cc:205] Tried to move back new op lower bound from 7316208218011561984 to 7316208217976954880. Current Snapshot: MvccSnapshot[applied={T|T < 7316208218011561984}]
I20260808 10:07:26.983306 25686 mvcc.cc:205] Tried to move back new op lower bound from 7316208218011561984 to 7316208217976954880. Current Snapshot: MvccSnapshot[applied={T|T < 7316208218011561984}]
I20260808 10:07:26.984827 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.968951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.24.162.188:39751)
0808 10:07:26.968986 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.968988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.975078 (+ 6090us) server_negotiation.cc:407] Connection header received
0808 10:07:26.975193 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.975194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.975206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.975262 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.976297 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.976977 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.980141 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.980438 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.980505 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.981913 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.981915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.981915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.981922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.983590 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.983990 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.983991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.983991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.984073 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.984277 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.984278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.984279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.984399 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:26.984486 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:26.984641 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:07:26.984728 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:26.992523 25524 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } attempt: 3
I20260808 10:07:26.994045 25681 catalog_manager.cc:1347] Generated new cluster ID: dd8353e1d809494285305efc0d56b35e
I20260808 10:07:26.994107 25681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:26 dist-test-slave-nfb5 krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.162.189@KRBTEST.COM for kudu/127.24.162.188@KRBTEST.COM
I20260808 10:07:27.000353 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.992838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.24.162.188:39751)
0808 10:07:26.992896 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:26.992898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:26.992909 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:26.992992 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:26.992993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:26.993003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:26.993051 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:26.993394 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.994085 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.995142 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.995440 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.995494 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.997428 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:26.997429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:26.997430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:26.997438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:26.999216 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.999544 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.999545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.999546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.999604 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:26.999822 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:26.999823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:26.999824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:26.999943 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:27.000000 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:27.000191 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 10:07:27.000256 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:27.000355 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:26.992792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.162.188:39751 (local address 127.0.0.1:59044)
0808 10:07:26.992868 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:26.992879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:26.992924 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:26.993061 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:26.993062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:26.993240 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:26.993285 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.993287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.994200 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.994201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.995036 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:26.995038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:26.995495 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:26.995496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:26.995553 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:26.996015 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:26.996020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:26.997337 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:26.999289 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.999290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.999290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.999464 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:26.999689 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:26.999690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:26.999690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:26.999752 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:27.000016 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:27.000017 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:27.000071 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 10:07:27.000153 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:07:27.000254 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 10:07:27.002354 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: Bootstrap starting.
I20260808 10:07:27.003113 25524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:27.003427 25524 log.cc:824] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:27.004276 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: No bootstrap required, opened a new log
I20260808 10:07:27.006356 25524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:27.006498 25524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72aacf4f312b4a9b880f42f84e92fe6a, State: Initialized, Role: FOLLOWER
I20260808 10:07:27.006667 25524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } }
I20260808 10:07:27.007174 25524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:27.007274 25486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:27.008237 25486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [term 1 FOLLOWER]: Refusing update from remote peer 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:27.008489 25487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72aacf4f312b4a9b880f42f84e92fe6a"
I20260808 10:07:27.008584 25634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:27.009474 25487 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bd7d4bdd12e41ee967c5cbfbdb2d8ab in term 1.
I20260808 10:07:27.011269 25681 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:27.012024 25681 catalog_manager.cc:1504] Loading token signing keys...
W20260808 10:07:27.012172 25710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:27.012217 25710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:27.012894 25694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:27.013008 25694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:27.013093 25694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bd7d4bdd12e41ee967c5cbfbdb2d8ab. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:27.013170 25694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:27.013229 25694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:27.013330 25694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:07:27.014073 25712 mvcc.cc:205] Tried to move back new op lower bound from 7316208218143166464 to 7316208217976954880. Current Snapshot: MvccSnapshot[applied={T|T < 7316208218011561984}]
I20260808 10:07:27.023093 25709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bd7d4bdd12e41ee967c5cbfbdb2d8ab" member_type: VOTER last_known_addr { host: "127.24.162.190" port: 45581 } } peers { permanent_uuid: "72aacf4f312b4a9b880f42f84e92fe6a" member_type: VOTER last_known_addr { host: "127.24.162.189" port: 45927 } } peers { permanent_uuid: "8be5133e86d0494cadbe3c4dce6c54a8" member_type: VOTER last_known_addr { host: "127.24.162.188" port: 39751 } } }
I20260808 10:07:27.023195 25709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:27.024389 25681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bd7d4bdd12e41ee967c5cbfbdb2d8ab: Generated new TSK 0
I20260808 10:07:27.024562 25681 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:07:27.051501 25419 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.050055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.24.162.190:45581)
0808 10:07:27.050102 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.050104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.050787 (+ 683us) server_negotiation.cc:407] Connection header received
0808 10:07:27.050892 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.050893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.050904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.050972 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.051432 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54830: BlockingRecv error: recv got EOF from 127.0.0.1:54830 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 10:07:27.052217 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.050482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.24.162.189:45927)
0808 10:07:27.050521 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.050523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.051745 (+ 1222us) server_negotiation.cc:407] Connection header received
0808 10:07:27.051837 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.051838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.051847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.051875 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.052156 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49634: BlockingRecv error: recv got EOF from 127.0.0.1:49634 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 10:07:27.053175 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.052551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.24.162.188:39751)
0808 10:07:27.052590 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.052592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.052705 (+ 113us) server_negotiation.cc:407] Connection header received
0808 10:07:27.052793 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.052794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.052804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.052832 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.053111 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59046: BlockingRecv error: recv got EOF from 127.0.0.1:59046 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 10:07:27.055419 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.054680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.24.162.189:45927)
0808 10:07:27.054713 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.054715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.054896 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:07:27.055022 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.055023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.055043 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.055077 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.055373 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49640: BlockingRecv error: recv got EOF from 127.0.0.1:49640 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 10:07:27.058343 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.057587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.24.162.189:45927)
0808 10:07:27.057629 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.057630 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.057877 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:07:27.057969 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.057970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.057979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.058008 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.058301 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49642: BlockingRecv error: recv got EOF from 127.0.0.1:49642 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 10:07:27.062143 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.061342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.24.162.189:45927)
0808 10:07:27.061375 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.061376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.061619 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:07:27.061741 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.061742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.061752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.061782 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.062103 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49648: BlockingRecv error: recv got EOF from 127.0.0.1:49648 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 10:07:27.067981 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.067197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.24.162.189:45927)
0808 10:07:27.067229 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.067230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.067483 (+ 253us) server_negotiation.cc:407] Connection header received
0808 10:07:27.067576 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.067577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.067587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.067615 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.067940 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49654: BlockingRecv error: recv got EOF from 127.0.0.1:49654 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 10:07:27.078037 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.077109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.24.162.189:45927)
0808 10:07:27.077147 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.077149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.077432 (+ 283us) server_negotiation.cc:407] Connection header received
0808 10:07:27.077542 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.077543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.077554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.077584 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.077952 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49664: BlockingRecv error: recv got EOF from 127.0.0.1:49664 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 10:07:27.096357 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.095394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.24.162.189:45927)
0808 10:07:27.095439 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.095441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.095756 (+ 315us) server_negotiation.cc:407] Connection header received
0808 10:07:27.095861 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.095862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.095873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.095903 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.096301 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49676: BlockingRecv error: recv got EOF from 127.0.0.1:49676 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 10:07:27.130872 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.129957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49684 (local address 127.24.162.189:45927)
0808 10:07:27.130001 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.130003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.130272 (+ 269us) server_negotiation.cc:407] Connection header received
0808 10:07:27.130374 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.130375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.130387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.130417 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.130791 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49684: BlockingRecv error: recv got EOF from 127.0.0.1:49684 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 10:07:27.197046 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.196261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.24.162.189:45927)
0808 10:07:27.196336 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.196339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.196458 (+ 119us) server_negotiation.cc:407] Connection header received
0808 10:07:27.196507 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.196508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.196518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.196572 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.196970 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43658: BlockingRecv error: recv got EOF from 127.0.0.1:43658 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:07:27.328620 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.326304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.24.162.189:45927)
0808 10:07:27.326373 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.326376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.327908 (+ 1532us) server_negotiation.cc:407] Connection header received
0808 10:07:27.327994 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.327995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.328014 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.328082 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.328551 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43662: BlockingRecv error: recv got EOF from 127.0.0.1:43662 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:07:27.586565 25528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:27.585923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.24.162.189:45927)
0808 10:07:27.586015 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:27.586017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:27.586034 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:07:27.586058 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:27.586059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:27.586069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:27.586141 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:27.586514 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43670: BlockingRecv error: recv got EOF from 127.0.0.1:43670 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:07:27.971490 25671 catalog_manager.cc:1259] Loaded cluster ID: dd8353e1d809494285305efc0d56b35e
I20260808 10:07:27.971570 25671 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: loading cluster ID for follower catalog manager: success
I20260808 10:07:27.972491 25671 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: acquiring CA information for follower catalog manager: success
I20260808 10:07:27.972846 25671 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8be5133e86d0494cadbe3c4dce6c54a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:07:28.012639 25710 catalog_manager.cc:1259] Loaded cluster ID: dd8353e1d809494285305efc0d56b35e
I20260808 10:07:28.012723 25710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: loading cluster ID for follower catalog manager: success
I20260808 10:07:28.013660 25710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: acquiring CA information for follower catalog manager: success
I20260808 10:07:28.014004 25710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72aacf4f312b4a9b880f42f84e92fe6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:07:28.103104 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:28.100177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.24.162.189:45927)
0808 10:07:28.100308 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:28.100311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:28.100373 (+ 62us) server_negotiation.cc:407] Connection header received
0808 10:07:28.100479 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:28.100480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:28.100560 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:28.100661 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:28.100813 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.101424 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.102195 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.102416 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.102507 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:28.102697 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:28.102809 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:28.102901 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:28.102976 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:07:28.102997 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260808 10:07:28.107239 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:28.104867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.24.162.188:39751)
0808 10:07:28.105008 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:28.105011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:28.105019 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:28.105056 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:28.105057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:28.105094 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:28.105168 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:28.105273 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.105736 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.106406 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.106630 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.106681 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:28.106829 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:28.106981 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:28.107041 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:28.107104 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:07:28.107128 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260808 10:07:28.111724 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:28.109003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.24.162.188:39751)
0808 10:07:28.109038 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:28.109040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:28.109216 (+ 176us) server_negotiation.cc:407] Connection header received
0808 10:07:28.109320 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:28.109320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:28.109330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:28.109359 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:28.109522 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.110024 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.110952 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.111207 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.111292 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:28.111470 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:28.111501 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:28.111555 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:28.111617 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 10:07:28.111641 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:07:28.111931 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:28.108929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.24.162.189:45927)
0808 10:07:28.108965 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:28.108967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:28.109155 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:07:28.109218 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:28.109219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:28.109231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:28.109285 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:28.109469 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.110222 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.111092 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.111394 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.111480 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:28.111666 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:28.111697 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:28.111725 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:28.111845 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:07:28.111867 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:07:28.112583 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:28.108935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.24.162.190:45581)
0808 10:07:28.109054 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:28.109056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:28.109179 (+ 123us) server_negotiation.cc:407] Connection header received
0808 10:07:28.109293 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:28.109294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:28.109348 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:28.109437 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:28.109615 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.110370 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.111373 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:28.111706 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:28.111775 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:28.112226 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:28.112330 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:28.112411 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:28.112472 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:07:28.112493 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260808 10:07:28.114385 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25316
I20260808 10:07:28.127379 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25425
I20260808 10:07:28.135636 25226 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHQacpx/build/release/bin/kudu with pid 25532
2026-08-08T10:07:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3019 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6048 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281423 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST